Fitting your bicycle to your body and needs can range from a very general "close enough" for the casual around-the-neighborhood cyclist, to a detailed and exacting process utilized by racers and other high-performance riders. If you already have some miles under your belt and need help fine-tuning your fit, I recommend Jim Langley's bike fit page.
For those who might be a little intimidated about getting back on the bike, read on...
A common concern I hear from novice cyclists is, "But I want to be able to put my feet flat on the ground when I stop." With a conventional bicycle a saddle that low means your legs are not getting full extension and you're not getting full use of your leg muscles. Not knowing how to start and stop the bike properly is part of the reason novices are uncomfortable with the proper saddle height. Here is a video demonstrating proper starting and stopping.
Proper leg extension is not only a performance matter, riding with inadequate extension can lead to knee problems.
If putting your feet flat on the ground is very important to you, then I recommend you look at one of the many "crank forward" bikes available, such as the Electra. These bikes are designed so you can still have proper leg extension while also being able to put your feet flat on the ground. The down side to this is that your very upright position will catch a lot of wind, so you won't be able to go very fast.
Make sure the bike fits by straddling the top tube and lifting the handlebars straight up. For a road bike with a top tube parallel to the ground you should be able to lift the front wheel one to two inches off the ground. If it's a mountain bike, a hybrid or comfort bike, or a road bike with a sloping top tube, you should be able to lift the front wheel two to four inches.
Your saddle should be high enough that your knee has about a 10- to 15-degree bend when your foot is at the bottom of the pedal stroke. Your feet should be positioned on the pedals so that the balls of your feet are directly above the spindles of the pedals. Having the balls behind the spindle for extended periods puts you at risk for stretched Achilles tendons. Toe clips help to keep your feet in the correct position (but they must be the right size) and also keep your feet from slipping forward off the pedals; "clipless" pedals do the same. Your saddle can also move forward and back. As Jim Langley writes: "The fore/aft seat adjustment is correct when a plumb line (any piece of string with a weight on the end) hanging from the bony protrusion just below your kneecap, bisects the pedal axle."
While Jim Langley's bike fit system is very precise, you don't have to follow all of it if you don't have plans for racing or longer rides. Notice that the fitting generally starts at the feet and legs and works its way up your body to your arms and hands. Because they are your motors, and because a bad fit can harm your knees or Achilles tendons, the fit of the feet and legs are most important. If you're not concerned with going as fast as possible (which means crouching over to get "aero"), you may wish to slack off on some of the upper body positioning; perhaps to sit a bit more upright.
Don't try to adjust your saddle position to shorten your reach to the handlebars (see Fore/Aft Seat Position on Jim's page). Get the saddle positioned correctly for your legs, then if your reach to the bars is too long or short, get a shorter or longer stem (or an adjustable one) to give you the reach you'll be comfortable with.