France's defence ministry said Dutch hostage Sjaak Rijke, kidnapped in Timbuktu in November 2011, was freed during "military action carried out by the French army's special forces."
"This combat action has also led to the capture of several individuals," added the ministry.
French President François Hollande also spoke of the operation, saying the freed hostage had been transported to a safe location and that a number of suspected jihadists had been killed.
He also signaled that French soldiers were not aware of the hostage's location before the raid against the extremists.
"Several hours ago, French special forces were able to carry out an attack against a terrorist group and were able not only to neutralise the group, but also obtain the freedom … of a Dutch man who had been held hostage since November 2011," Hollande told journalists.
He said the assault occurred near Tessalit in Mali's far north, near the border with Algeria.
Asked whether the hostage was in good condition, Hollande said "as much as possible after four years of captivity."
"It was a surprise for us — for our forces — to be able to free this hostage because we did not have information" on his presence, he said, without providing further details.
Dutch Foreign Minister Bert Koenders said Rijke was "doing well considering the circumstances."
"It is extraordinary news for Sjaak and his family," Koenders said. "I am happy and grateful that this horrible period of uncertainty and distress is over."
'Battle against terrorism'
Gunmen had stormed into Rijke's hotel capturing him as well as a South African and a Swede, both of whom are still being held.
In November 2014, AQIM Islamists released a video of Rijke making a statement on the 1,000th day of his captivity.
Also seen on the video was Frenchman Serge Lazarevic, who was later freed in a prisoner swap against four jihadists in Mali.
Mali's vast desert north is riven by ethnic rivalries and an Islamist insurgency.
Jihadists linked to Al-Qaeda controlled Mali's north for more than nine months in 2012 until they were routed by a French-led military intervention, but extremist fighters remain active throughout the restive northeast.
Mali is also struggling with ethnic Tuareg militants fighting the army over northern territory they claim as their homeland and call Azawad.
Hollande said "the battle against terrorism in Mali is not over. We still have 3,000 men in the region — not only in Mali — to carry out the battle against terrorism."
The troops he spoke of serve under the Barkhane operation targeting jihadists in the Sahel region.
France announced last month that it would further boost its military presence in the Sahel to also support the fight against Boko Haram.