Forest Fenn claims the deaths of two people who were hunting for the $2 million in gold are not his fault.
The eccentric millionaire who triggered a deadly treasure hunt for $2 million in gold is under mounting pressure to bring the search to an end.
Read: Treasure Hunter Dies While in Pursuit of $2 Million in Gold Buried in Rocky Mountains
Two men have died in remote areas of New Mexico while on a quest to find a chest of gold that millionaire Forest Fenn claims he buried somewhere in the mountains.
The latest victim was 52-year-old Paris Wallace, a pastor from Colorado. His death was announced earlier this week.
New Mexico Police Chief Peter Kassetas confronted Fenn on Good Morning America, imploring him to stop the treasure hunt.
"You had talked about giving more clues, providing more clues to help people better find your treasure and again, I call for you to pull it," Kassetas told Fenn over the phone.
Fenn replied: "I am not going to give a clue so people find the treasure. I am going to give a clue to keep them out of trouble to make them safer."
"I want Mr. Fenn to retrieve the treasure or call off the hunt," Kassetas told ABC News. “It's solely based in the interest of public safety."
Fenn is an 86-year-old art and antiques dealer who made millions in the 1980s while selling Indian artifacts.
An amazing 65,000 people are believed to be looking for the buried treasure in a vast area stretching from Montana to New Mexico and they are working from tantalizing clues contained in a poem in Fenn's 2011 book, The Thrill of the Chase.
Fenn told Inside Edition it's not his fault people are risking their lives, saying, "In the summertime we jump in the swimming pool. If someone drowns in the swimming pool should we drain the pool or teach people to swim? That's the way I feel about it."
But Linda Bilyeu, whose ex-husband Randy died last year while searching for the gold, says she doesn't believe there is any treasure.
Read: Beachcomber With Metal Detector Finds Woman's Wedding Ring She Lost While on Vacation
“I think this whole thing is a hoax. There has been no proof whatsoever that the treasure actually exists,” she told Inside Edition. “I think he should end the search. Produce the chest or say it's a hoax do what he has to do just to say it's over. There comes a time when you have to just let it go.”
But Fenn claims the gold is real, telling Inside Edition, “There is a treasure out there.”
Fenn says he is now speaking to friends about reevaluating the treasure hunt.
Watch: Father and 5-Year-Old Son Found Dead in Lake After Failing to Return From Hunting Trip