The basic legal requirements for starting a business are the same no matter where you start your company.  Whether you live in California, or New Hampshire, you will want to follow the same checklist.

The Small Business Administration website is an excellent resource for just such a checklist. 
This article will highlight certain steps of the SBA's checklist that you should consider before starting a business.

For example, step number 5 of the checklist is to determine the legal structure (the company form) of your business.  As discussed in the article company forms, management, liability, and profits all vary between forms of companies.  Company forms are also taxed differently from state to state.  In fact, even which company forms are legally sanctioned vary from state to state. 

Other steps to consider are 8, 9, 10 for the same basic reason as step number 5.  State laws may affect tax, licensure, and employer responsibilities.  For example, it would be a mistake to assume that just because Delaware has corporate friendly laws that other nearby states will as well.

Also, it should be stressed that the sba.gov checklist does not have to be followed in the exact order presented.  You may know from previous experience that you want a Limited Liability Corporation (LLC) (step 5 of the checklist) long before you ever get around to writing the business plan for it (step 1), or that you want to establish a business in Nevada due to business friendly tax laws (step 8) before you know exactly where you are going to set up shop (step 3).

Bottom line: do some research and determine your priorities before processing the SBA.gov checklist.  Use it as a means to verify that you have met all the legal requirements for starting a business.