The prime minister wants independent senator Fraser Anning to face "the full force of the law" over his comments the Islamic community was to blame for the New Zealand mosque massacre.
And New Zealand's representative in Australia says the Queensland senator's comments were atrocious.
The government and Labor are planning a bipartisan motion condemning Senator Anning when parliament returns in April.
However, Scott Morrison went further on Sunday, when asked if a parliamentary motion was enough, saying: "I think the full force of the law should be applied to Senator Anning."
He previously said the comments the senator made were appalling, ugly and had no place in Australia.
New Zealand High Commissioner Annette King said her country rejected absolutely Senator Anning's sentiment.
"We just condemn what he said and I'm absolutely delighted that his comments were condemned from the very top of government here in Australia and by the people of Australia," she told reporters in Canberra.
However, she said it was up to Australians to decide if he was fit to be an elected representative.
Government Senate leader Mathias Cormann and his Labor counterpart Penny Wong will jointly move the motion to censure Senator Anning "for his inflammatory and divisive comments ... which do not reflect the opinions of the Australian Senate or the Australian people".
They will ask the Senate to recognise that "an attack on any religion is an attack on all religions and that we all share a responsibility to unite, condemn and defeat such an attack on our common values and way of life" and call on all Australians to publicly condemn actions and comments designed to incite fear and distrust.'
A change.org petition calling for Mr Anning's expulsion from parliament has amassed almost 815,000 signatures.
A 17-year-old boy was arrested after allegedly throwing an egg at Senator Anning at a speaking event at Moorabbin in Melbourne on Saturday.
The teenager has since been released pending further inquiries.
He posted a video on social media saying: "Don't egg politicians, you get tackled by 30 bogans at the same time - I learnt the hard way."
Senator Anning used his Facebook page to thank those who had supported him after the incident.
He made several more anti-Islamic posts and complained about "the backlash of the establishment media and globalist politicians".
He also said Twitter had banned him from posting there for breaching the platform's community standards.
Senator Anning came to the Senate in late 2017 as a replacement for One Nation's Malcolm Roberts after he was found to have been a dual citizen and therefore ineligible for election.
While Senator Anning ran on the One Nation ticket at the 2016 poll, he quit the party minutes before being sworn in after falling out with leader Pauline Hanson.
He was roundly condemned after invoking the phrase "final solution" in his first speech to parliament - a reference to the Holocaust - and again after attending a far-right rally in Melbourne where participants made Nazi salutes.
Senator Hanson - who has previously described Islam as a "disease we need to vaccinate ourselves against" - wrote on Facebook that while the shooting spree in Christchurch must be condemned, she was concerned that "the damage caused to conservative views on immigration by alleged gunman, Brenton Tarrant will drive debate underground".
Australian Associated Press
