The San Francisco was travelling from the Philippines to Mexico when it sank.
The galleon was believed to be carrying valuable trade goods which could be worth millions today, researchers say.
Its location has been a mystery - but the suspected cannonball, thought to be the first artefact ever found from the ship, offers clues about where it sank. Dr Jun Kimura from Tokai University has been leading a team of maritime archaeologists, who have been searching for the San Francisco in waters off Iwawada in Chiba prefecture.
The cannonball was discovered by Ian McCann, an Australian researcher at the University of New England, during a deep dive nearly 40m (131 ft) below the surface.
"We were in dark, murky waters," Dr Kimura told the BBC. "Ian just saw an unusual shape on the sandy bed - he recovered it but then we had to go back to the surface as our air had nearly run out."
He said the team, and archaeological experts they had consulted, were "almost certain" it was a cannonball from the San Francisco, as it was similar to cannonballs found in other Spanish trading ships in the Philippines.
However, they will be carrying out a chemical analysis to confirm this.