Vendors are typically in view of the music, so attendees can shop and not miss a note. Merlefest used to spread vendors out, but for 2007, it created "The Shops at Merlefest," a concentrated area where vendors rent space for a fl at fee. Cloeren has 75 vendors in a smorgasbord of clothing, pottery, crafters, memorabilia, art, etc, and—unusually—has designed the traffic flow so guests go through the vendor area, on their way between alternating main stages. Guests have 15 minutes between acts to get out of their seats, walk through the vendors, and bam! they're back in their other seats at the other main stage.

Pocono rents stages, sound, lights and backline reinforcement. The rest is all in-house—management, merchandise, security, first aid, food and beverage and real bathrooms. Mazzolini, full-time and without staff , books the hotels, the performers, does fundraising, writes the bios. He subs the specialized things going in to the festival jobs, and 98% get paid, without a significant volunteer population. Of the 80 to 100 people on site, many have worked there for the last 30 years. Too many volunteers get in each other's way and some will disappear into the crowd, just getting in for free.

Powell is one of 70 year-round staffers working on booking, advertising, sponsorship, etc. Lollapalooza is the biggest rock 'n' roll production in the USA and ACL is up there. Just from a production standpoint, securing the fenceline and bringing in the massive stages are monumental tasks. Powell deals with subcontractors all over the country—Atlanta, Las Vegas, NYC—everything at the festival is outsourced—100 stagehands, food vendors, backline companies, an experienced concert security company that's used to handling these events, plus 300 volunteers per festival. Dolin, at Chicago, has 60 people ongoing in the office, and 180 on site.

"The attitude of security, and presenting a safe and orderly day for everybody" is paramount, according to Lang, who learned his hardest lesson with a death at Altamont. Not surprisingly, Lang now recommends private security companies accustomed to dealing with concert audiences.

Gustafson hires a professional team of Samoans for crowd control. "I'd cancel a band before I'd take the Samoans out," he said. Once, when someone rushed the stage, "those Samoans grabbed him by the ankles and the guy vaporized. We don't know what happened to him. He's somewhere out in the sagebrush. They are nice guys, but they take care of business."

If the tone is right, however, few problems arise. The Rockland, Maine, police chief surveyed 15,000 people and told Benjamin "we don't need to be here." Powell said "We try to make it an enjoyable experience, so people are in a good mood, not angry. In five years of ACL and two years of Lollapalooza, we've never had one arrest." Both Lang and Powell cited cheap water as a factor, and overall good quality, properly priced. At one festival, in an economically depressed area where local farmstands sell corn for $1.25 a dozen, a vendor offered roasted corn on the cob at $3.00 an ear. Attendees weren't happy.

An interesting mix of music and multiple stages helps maintain the mood. If someone doesn't particularly like one band, they have the option of hearing another rather than getting bored, fidgety, or high. Dean notes that with 12-15 bands, a tweener stage with acoustic stuff keeps people entertained. "If it takes 20 minutes to set up, people's attention spans are so short they'll wander off ." At AmsterJam, Lang created a Village Zone with alternatives to sitting in front of a stage, or additional stages for those not into that type of music. Bringing audiences and bands together enhances the vibe. Benjamin puts 400 chairs up close, first come, first served: "My front row touches the stage."

At Pocono, it's not just a gig for the musicians, they're fans too, and they're encouraged to hang out in the crowd, and they do.

Plenty things can go wrong, big-time, and a promoter's ability to effect damage control is crucial. Hands down winner of "What's a promoter's greatest fear?" That nobody shows up. "I've had that fear since Day One," Dolin admitted.

The second, more realistic, concern is of people getting hurt. That is tops on Lang's list. Benjamin worries that someone gets hurt or arrested on his club crawl, or a power outage, "because there's not a damn thing you can do about that." Not exactly. In 2002, for the first time in 100 years, a tornado was predicted to hit NJ on Dean's festival Friday, but actually hit Thursday. "We thought we were so lucky, then found out we had no electric," Dean recalled. Friday morning, her partner drove in with five generators in his truck, just as the electricity kicked on. "I heard the refrigerator start up, and throughout the campground you could hear people going 'Yeaaa!'"

Weather, of course, is on everyone's mind, and though festivals proceed rain or shine, bad weather can disastrously delay setup and depress attendance. Pinedale's weather insurance "is having the festival in August, not June." Boomer costed out weather insurance and decided to buy a bond instead. Lang is philosophical, and resigned: "I know it's going to rain and I know I can't get insurance." Powell tracks the weather six weeks out.