"Everything has gone. Everything. Including door hinges and a door knob, everything made of metal has gone," said Shusaku Yoshida, the 67-year-old caretaker of a shelter facility for local fishermen.
Some of the North Korean fishermen have admitted to "taking out" some things from the small island, according to Jiji press.
Japanese authorities are continuing to question nine of the 10 North Koreans first spotted last week aboard a tiny wooden boat struggling in bad weather off the northern island of Hokkaido.
The other was reportedly sent to hospital on Monday with a stomach ache.
The fishermen - who identified themselves as North Korean - briefly landed on an uninhabited island to take refuge.
But soon after the group left, officials noticed items from buildings on the island had vanished.
The caretaker said the shelter's lock had been broken and gave a long list of items missing from inside the lodge.
"Two TV sets, three fridges, a washing machine, an oven/microwave, two stereo sets, a DVD/CD player, an electric saw, a stove (heater), coal, a motorbike and a generator," he told private broadcaster Fuji TV.
"I was so surprised, I had to sit down," he said.
A coastguard official said on Monday that the list of items that had disappeared included some of the solar panels at the lighthouse on the island, an anime poster and blankets.
More than 60 boats from North Korea have already been discovered this year in Japanese waters, according to the coastguard.
Experts say some North Korean fishermen are travelling far out to sea in order to satisfy government mandates for bigger catches.
But their old and poorly equipped vessels are prone to mechanical and other problems, including running out of fuel, and there are few ways for them to call for rescue.
KEEP UPDATED on the latest news by subscribing to our FREE newsletter.
- FOLLOW News24 on Twitter