Following the events of Heroes Chronicles: The Sword of Frost, Gelu, the leader of AvLee's Forest Guard and wielder of Armageddon's Blade, attacks Kilgor, the Barbarian King of Krewlod, in an attempt to claim and destroy the Sword of Frost. As foretold by a prophecy, when Armageddon's Blade and the Sword of Frost clash in battle, the result is a massive explosion (known as the Reckoning), which destroys the world of Enroth. Many of the inhabitants of the planet, however, manage to escape through mysterious portals that appear during the Reckoning which lead them to another world, called Axeoth - also the setting for Might and Magic IX. Included among the refugees are many of the heroes of previous installments of the series, and many return to their old ways shortly after arriving on their new home.

The story of Heroes of Might and Magic IV unfolds primarily through the game's six campaigns, each of which establishes how the major kingdoms of one of Axeoth's continents were forged in the wake of the Reckoning, and the arrival of the Enrothian refugees on Axeoth. Each of the campaigns centers on a faction leader, and tells the story of how that leader came to prominence. Unlike Heroes of Might and Magic III, all of the six campaigns are accessible from the beginning, and each unfolds as a stand-alone tale.

In the Haven campaign, The True Blade, Lysander, a loyal knight formerly in the service of Queen Catherine Ironfist, rules over the nascent kingdom of Palaedra, inhabited mainly by refugees from the kingdom of Erathia. His command, however, is challenged by a usurper named Sir Worton, who claims to be the only surviving heir to the Gryphonheart dynasty. Many of Lysander's generals are drawn to Worton, forcing the lord to expose Worton as a fraud or lose control of his kingdom.

At the start of the Stronghold campaign, Glory of Days Past, Waerjak, a barbarian chieftain, despairs over the changes that have come over the barbarian community. In the wake of Kilgor's death, his people have descended into interfactional strife, and are in danger of wiping themselves out. Intent on restoring the barbarian kingdom to its more peaceful state before Kilgor's ascension, Waerjak sets out to prove his strength and to subdue his rivals, guided by his mentor, Tarnum (previously the protagonist of the Heroes Chronicles series).

The game's Academy campaign, The Price of Peace tells the story of Emilia Nighthaven, the peasant daughter of an Enrothian glassblower. On Axeoth, however, she finds herself taking command of a community of frightened refugees and forging them into the kingdom of Great Arcan. Queen Emilia's success, however, attracts the attention of Gavin Magnus, the Immortal King of Bracada from Heroes of Might and Magic III and Might and Magic VII, and his genie servant, Solmyr, who seek to prevent Axeoth from meeting the same fate as Enroth (i.e. being destroyed) by magically depriving its entire population of their free will.

In the Preserve campaign, Elwin and Shaera, Elwin, a simple elf from the Elven kingdom of Aranorn (mainly populated by survivors from AvLee), is in love with Shaera, but their romance is threatened by Lord Harke, a powerful rival for Shaera's affection. Elwin's continued pursuit of his beloved plunges the region into internecine war, and whoever emerges the victor will not only win Shaera's hand in marriage, but the throne of Aranorn, as well.

Half-Dead, the Necropolis campaign, relates the tale of Gauldoth Half-Dead, the half-human, half-lich victim of a necromantic spell that he cast to save his life which went horribly wrong. Tired of living on the scraps of food he can steal from fearful villagers, Gauldoth unites the forces of the surviving Necromancers and demonic Kreegans from Enroth's old kingdoms of Deyja and Eeofol into Nekross, a powerful and feared kingdom, but he is forced to act as a protector for both the living and the dead when a powerful being from another dimension arrives with plans to end all life in the universe.

A Pirate's Daughter, the game's Asylum campaign, introduces the only daughter of a famous pirate, Tawni Balfour. Tawni inherits her father's ship and crew following his unfortunate death. Sailing down the coasts of the Gold Sea, fighting feared buccaneers, sea monsters and mermaids alike, Tawni intends to claim her father's mantle as Axeoth's most feared pirate captain, all while having to face her own shady past.