All paper, except tissue and shredded paper, can be recycled.
You do not need to remove envelope windows.
You may like to remove personal details from letters and envelopes, tear off the address and recycle the rest.
If shredded, this can make an excellent addition to your home compost (if they are plastic free).