Sierra Madre is historically linked to the old mountain resorts of the San Gabriel Mountains and Valley. The Sierra Madre Villa Hotel was a pioneer of summer resorts that populated the San Gabriel Valley in the late-1800s. Adams' Pack Station, located within the 91024 zip code at Chantry Flat, is the last remaining pack station in Southern California. The municipality also operates maintains the landmark Lizzie's Trail inn at the head of Old Mount Wilson Trail.

The Sierra Madre Villa Hotel, 1884.

Sierra Madre is known for its annual Wistaria Festival (an alternative spelling of Wisteria), which celebrates its 114 year old, 1+ acre Chinese wisteria (Wisteria sinensis) vine. The plant was named by the Guinness Book of World Records as the largest blooming plant and one of the seven horticultural wonders of the world. The annual festival is the one day a year the 115 year old vine on private property can be viewed.

The town is also home to the only remaining volunteer fire department in the Greater Los Angeles area and has no traffic signals within its town limits. With the appointment of former Pasadena Police Department Cmdr. Marilyn Diaz as Police Chief on March 14, 2006, the Sierra Madre Police Department became the first municipal police department in Los Angeles County to have a female Chief. As reported by KCAL television in 2006, there are currently 17 sworn members of the Sierra Madre Police Department.

This small village in the Sierra Madre Canyon area is noted for being a distinct and unusual community. The Canyon is characterized by narrow and winding roads, lush vegetation, views of the San Gabriel Valley, and small bungalows or cabins. The downtown has small restaurants and shops.

There is also a historic company that makes gourmet and specialty olives, jams, jellies, and syrups (sold to the public) from its own citrus groves. The company, E. Waldo Ward and Son, was founded over 120 years ago, when most of Sierra Madre was mainly agricultural in its zoning; the first Sevilla orange trees in the U.S. were planted on the grounds (now, due to re-zoning, the original trees are on private property, however the Ward family still owns 3 acres (12,000 m2) of oranges at 273 E. Highland Ave. as well as having a barn.)

Sierra Madre host a 4th of July Parade each year.