NEW DELHI (CNN) -

A group of locals looked on warily as we filmed our piece inside a congested, riverside Delhi neighborhood.

And they've every reason to feel nervous. A number of deadly collapses in recent months have thrust the issue of illegally constructed and unsafe housing into the spotlight.

On June 28, a four-story building in the city suddenly toppled over killing 10 people, including five children. That same day, an eleven-floor building under construction came crashing down in the southern Indian city of Chennai, killing 61 workers. Both incidents made headlines outside India -- and rightly so.

But for many of the people living in these neighborhoods, the publicity is not necessarily welcome. That's because they're among the poorest and most vulnerable -- on one hand they fear for the safety of their homes, and on the other they fear being forcibly evicted and their homes destroyed. They have nowhere else to go.

So when TV crews arrive, many are understandably defensive.

As we filmed in Laxmi Nagar, a neighborhood of eastern Delhi located along the Yamuna River, pockmarked with ramshackle residential blocks, a man approached us and questioned whether we had the right to film.

"You know, everyone gets worried when they see cameras pointing at their buildings," he explained. "As the leader of this market association, we have to inquire about the purpose of your shoot."

Safety codes ignored

This part of the city is no stranger to disaster. In 2010, at least 65 people were killed when a building came down after its foundations had been weakened by rainwater.

It's not just Delhi; decrepit constructions that don't conform to safety codes are a big problem in cities and towns across India, says Chandan Ghosh, from the Indian home ministry's National Institute of Disaster Management. He has surveyed Delhi's structures for decades, and says most break basic building regulations. He says the problem is so pervasive that enforcement has become impossible.

"Our systems are not equipped to enforce engineering standards required to build safe houses. There's a gross shortage of manpower, expertise -- and maybe will."

The Delhi residential block that caved in on June 28 was built on a plot barely 19 square yards. With no pillars or beams to support its floors, it collapsed after neighbors starting removing earth to re-build their own house.

Safety codes prescribe at least 150 square yards of land is required to construct a standard apartment, explains Ghosh.

Buildings in Indian cities are often unplanned and strung together in cramped rows, without space between them. Experts say this can be disastrous if one is more structurally vulnerable than the others. Owners of buildings often sell off parts of these buildings to individuals, who then construct extensions.

"This type of housing is not unauthorized but remains extremely dangerous," says Ghosh. "Unfortunately, it's mushrooming all over without checks and balances."

'Disaster waiting to happen'

With rapid urbanization occurring since India liberalized its economy in the 1990s, New Delhi's population has grown exponentially to almost 17 million -- more than 21% in the past decade, census figures show.

To make matters worse, Indian authorities believe the Indian capital is vulnerable to seismic activity.

In 1999, an 6.5 magnitude earthquake some 173 miles (280 kilometers) from Delhi caused some damage to buildings in the city, but raised fears about the potential harm from another temblor of similar intensity happening nearer. It's a "disaster waiting to happen," according to Ghosh.

According to Delhi's disaster-management department website, "Pockets with high-rise buildings or ill-designed high-risk areas exist without specific consideration of earthquake resistance. Similarly, unplanned settlements with sub-standard structures are also prone to heavy damage even in moderate shaking."

Yet people have to continue to put themselves at risk because they've nowhere else to go.

"A lethal cocktail of unethical constructions, substandard material (and) a lack of enforcement ... are endangering countless lives," says Ghosh.

"These fatal incidents happening every now and then are a warning. It's time all stakeholders, including citizens, took it seriously."

Meantime, two injured survivors mourn their loved ones at the site of last month's Delhi collapse.

His head bandaged, Naeem recalled how he literally rode the wave of debris when he tumbled four floors down. Underneath the rubble were his wife and three children, who all died.