Prime Minister Scott Morrison insists his position on constitutional recognition of Australia's indigenous people has not changed and hopes to make progress in a "good faith way".
But says he does not want to raise expectations of what can be achieved.
"I am a constitutional conservative on these issues, which should come as no surprise," he told reporters after addressing the state LNP convention in Queensland on Saturday.
"I am not going to raise peoples expectations on this, I am going to be very clear about where we are and provide a space, which we have done ... that can hopefully see this progress."
Indigenous Australians Minister Ken Wyatt committed to holding a referendum on constitutional recognition within the next three years this week.
But he and the prime minister will not support a constitutionally enshrined indigenous voice to parliament, as proposed in the Uluru Statement from the Heart.
Conservative Liberals and Nationals have also raised concerns an indigenous advisory body could become a "third chamber" of federal parliament, with one MP threatening to campaign on the "no" side of a referendum..
Liberal MP Trent Zimmerman concedes it is going to be hard to find the right model for constitutional recognition but it will be a process that does involve more consultation.
"What I'd encourage everyone to do is to sometimes 'sit back and take a breath'," he told ABC television.
"What we don't need is a knee-jerk reaction to every proposal or proposition that comes along."
He is also conscious of the tricky road that constitutional referendums have had in Australia.
"The majority of Australians - anyone under 40 - will never have seen a successful referendum passed in the country, which gives rise to a little bit of caution," he said.
Labor opposition frontbencher Amanda Rishworth says any discussion has got to start from the principle that Australia's first people got together and put forward the Uluru Statement from the Heart.
"We do have to respect that," she told ABC television.
"In saying that, I think we need to keep an open mind. We want to achieve this. And so we need to go forward with making sure that people aren't closing off to where we can get to with this."
Mr Morrison says he is focused on young indigenous people committing suicide in remote regional communities and making sure enough kids are turning up at school every day to get the education to set them up for the future.
"It's these very practical issues that are my top priority," he says.
Australian Associated Press