The Olympics could be held behind closed doors according to Tokyo 2020 president Seiko Hashimoto.
Hashimoto says the Games can only be a success if organisers "completely protect" athletes and people in Japan, and that she is hopeful people will "be glad" the event went ahead.
Overseas spectators have already been barred from the Games, and this week a decision on domestic fans was delayed until June, with organisers citing a new wave of infections in Japan.
"There might be a situation where we can't allow any spectators to attend," Hashimoto told AFP.
"The only way that we can call the Games a success is if we completely protect the lives and health of athletes and the people of Japan."
As it stands, most people in Japan back either a further delay of the Games or an outright cancellation, and a recent surge in virus cases has prompted a state of emergency in Tokyo and other parts of the country.
With the medical system already under pressure, Olympic organisers have been criticised for requesting volunteer medical staff for the Games.
Hashimoto said cutting spectators could ease pressure on the medical system.
"If the event itself does change, I think it will be with regard to spectators," she said.
"That is one area where we might be able to reduce the anxiety of people who are worried about the medical system."
The Games have never been cancelled outside of wartime and organisers have made it clear that they see no possibility of either further delay or a cancellation.