We were thinking about switching to solar power for heat? How much and where to start?

We have forced oil which is not allowed by our loan we received on our new house which is 200 years old. The house is currently 1,300 square feet. Instead of switching to gas, we were interested in solar power. How to I begin the research on this and where to buy?
Also how $$ is doing something like this? We don’t know anyone with solar power so any comments or suggestions would be great.
IN NJ!!!!!!!!!!!!!!!!!!

How hard and expensive is it to install solar panels on a house that is already built?

I live in central Texas. I want to install some solar panels on the roof of my house for electricity. Im just not sure how expensive it will be or if it even makes sense to do. My house is one story 2100 square feet.