Out of 134,000 cases, about 94,000 are still alive and some 40,000 have died and payments could be made to estates.
The data was released as part of a National Audit Office (NAO) report on the scandal. The report found the biggest error was an underpayment of £128,000, it added errors go back as far as 1985.
About one in eight arrears payments so far is for £40,000 or more. The Department for Work and Pensions (DWP) still has no agreed plan for dealing with deceased cases.
Even in the ‘correction' exercise, DWP staff were initially getting about an eighth of cases incorrect.
The DWP currently has 1.4 million "open tasks" on the customer database, some dating back to 2005.
The report credits former pensions minister Steve Webb, now a partner at LCP, with helping to bring the scandal to light. In May 2020, LCP published a paper entitled ‘Are thousands of older women being short-changed on their state pension', which was based on a freedom of information reply obtained by Webb in February 2020.
However, the DWP did not regard this as a serious problem until it undertook its own scan of records in the summer of 2020, and it made no provision for underpayments in its departmental accounts which were prepared in May/June 2020.
The former minister said: "This report highlights the fact that DWP failed to act over a period of many years when errors were found in state pension assessments.
"Tens of thousands of married women, widows and the over 80s have been underpaid, with arrears in some cases exceeding £100,000. It is very worrying that errors are still being made as part of the correction exercise, where the highest standards of quality control should be in force.
"DWP also needs to do everything it can to track down the families of pensioners who have sadly died and never received the pension they were due."
Webb called for "full transparency" from the government department about the correction process, with regular updates and a full explanation of exactly which cases are being reviewed.
"It should also explain how these errors were allowed to go on for so long and what lessons have been learned," he added.
The errors refer to married and divorced women as well as those over 80 who were entitled to uplifts in their state pension, explained Hargreaves Lansdown's Helen Morrissey.
Some women retiring under the basic state pension system were entitled to an automatic uplift in their state pension when their husbands reached age 65. But in many cases, this did not happen, and women had to actively claim it before the uplift was applied.
Other women have missed out on state pension uplifts because they did not tell DWP about their divorce.
The senior pensions and retirement analyst said: "There is an expectation that the state pension you receive is what you are entitled to but in this instance, this was clearly not the case. A combination of manual error and a complex system means thousands of women have missed out on payments that could have made a real difference to their standard of living in retirement.
"Many of these women had no idea that they were even entitled to a higher pension and so did not ask about it and while the DWP has begun making repayments it is likely going to take some time before all those affected have been identified. Anyone who believes they might be affected should contact the Department for Work and Pensions."
Morrissey added: "While this issue affected women on the basic state pension which was replaced in 2016 it still serves as an important reminder to make sure you and your loved ones receive what you are entitled to."
AJ Bell head of retirement policy Tom Selby said it was a "national scandal".
"Tragically, of the 134,000 people underpaid by an average of £8,900 each, around 40,000 are estimated to have died before being compensated.
"Furthermore, those lucky enough to still be with us may have been living in penury when they should have been enjoying their retirement.
"The 94,000 people still alive who are owed money by the DWP need to get it back as soon as possible. They also deserve answers about how this was allowed to happen over such a long period of time."
He added: "Once compensation has been paid, the government needs to undertake a comprehensive review of its processes to ensure these mistakes are never repeated.
"Trust in pensions is fragile at the best of times and failures such as this will not help. Sadly, it will likely take years, if not decades, to rebuild the confidence lost as a result of this scandal."