Senior Tory and Reform politicians have clashed over immigration policy in the Oxfordshire village of Piddington after the government approved plans to convert a defence site into accommodation for asylum seekers.

Zia Yusuf and Chris Philp raised their voices and spoke over each other in front of TV cameras and local people for more than three minutes. Yusuf urged his fellow home affairs spokesperson to apologise for his record as a minister. Philp said they should instead "stand in solidarity with residents" and Yusuf should "read the room".

The clash came hours after villagers said they would launch a legal challenge to plans to house more than 1,200 detained asylum seekers near them.

It emerged on Thursday that Angela Rayner, the housing and communities secretary, had approved full planning permission, a decision welcomed by the Home Office, which wants to end the use of asylum hotels.

In their clash, Yusuf said: "Chris, would you like to apologise for the things you did in government to bring hundreds of thousands of people into the country illegally?"

Philp, who served as an immigration minister, said: "That is disingenuous, Zia, as usual … I hope today we can stand in solidarity with the residents. Listen to the people clapping and read the room."

As the two men's voices grew louder, Philp said: "You are in a hole, stop digging."

Downing Street has defended the plans to convert a military base near a tiny Oxfordshire village into asylum-seeker accommodation, insisting the safety of the public would always be its "top priority".

Tim McNally, the parish council chair, told BBC Radio 4's Today programme there would be "many actions" by villagers, and that ministers should "absolutely" be braced for a legal challenge, which would need to be lodged within six weeks.

Ministers have faced opposition over the plans from villagers, who even made a symbolic attempt to declare the village independent from the rest of the country after a referendum of its 350 residents. Asked how the village would respond, McNally said: "There's going to be many actions, you will see. We will disclose that as we move forward."

Campaigners could also apply the new Hillsborough law, which is being considered by parliament. Once passed, the law would create a legally enforceable duty of candour that compels public officials and authorities to act transparently when investigations and inquiries take place.

"Today is a day of reflection, and we'll work out where we plan to go from this point. But I will say, the Hillsborough law is there for a reason, and we're going to apply it," McNally said.

A Home Office spokesperson said a final decision to proceed had yet to be taken. "The government is working on a fairer distribution of asylum accommodation across the country, with over £1bn saved in costs since the election," they added.

"This includes closing all asylum hotels and moving to alternative accommodation, including former military sites. These are designed to be self-contained, with 24/7 security and CCTV, to minimise the impact on the local community."