Hiroshima-jo was built by Mori Terumoto in 1591. Mori Terumoto was a very powerful daimyo who controlled the vast majority of the San'in and San'yo areas. Having outgrown his Yamato-Koriyama castle he built a new castle at Hiroshima in 1599. Mori Terumoto aligned himself with the Western forces in the Battle of Sekigahara (1600). The Western forces lost and Mori's lands were confiscated. He was then appointed to govern most of what is modern day Yamaguchi prefecture.

One of Hideyoshi's former allies, Fukushima Masanori filled Terumoto's place at Hiroshima. After Fukushima was stationed in Hiroshima, it's said that he got nostalgic for his old days under Hideyoshi. Needless to say, this did not sit well with the Tokugawa. In 1617 a great flood caused much damage to Hiroshima-jo. Fukushima petitioned the Tokugawa government for permission to fix it but he never received an answer. According to Tokugawa law, all daimyo needed permission to build, rebuild or renovate any castle. The only reply Fukushima got from the Tokugawa was "under investigation" and permission never came.

Two years later he proceeded on his own and started to fix the flood damage. Fukushima was caught in a Tokugawa trap. Since he defied their laws, they took away his lands around Hiroshima and gave him a smaller, less profitable province in modern day Nagano prefecture.

Hiroshima-jo lasted through the Meiji Restoration and was named a National Treasure in 1931 only to be destroyed by the atomic bomb in WWII.