In April, the court said a 2009 decision to drop the 783 charges against Mr Zuma was irrational.
National director of public prosecutions Shaun Abrahams had asked the court for permission to appeal against the earlier ruling, saying prosecutors should be able to exercise discretion over whether an investigation should continue.
Rejecting the request, High Court Judge Aubrey Ledwaba said: "We seriously considered whether the appeal would have reasonable prospects of success and came to the conclusion that there are no merits in the arguments."
The ruling is a blow to Mr Zuma's efforts to avoid standing trial, reports the BBC's Milton Nkosi from Johannesburg.
The president can still take the case to the Supreme Court of Appeal but it is difficult to see it overturning the unanimous ruling of the High Court, our correspondent says.
Mr Zuma's final option would be to ask the Constitutional Court to throw out the case.
Controversial arms deal: What you need to know
1999: largest-ever post-apartheid arms deal announced, with contracts totalling 30bn rand ($5bn; £2.5bn) to modernise national defence force
Deal involved companies from Germany, Italy, Sweden, the UK, France and South Africa
Allegations of bribery over deal dogged governments of President Jacob Zuma and predecessor Thabo Mbeki
Mr Zuma's former financial adviser Schabir Shaik convicted in 2005 for corruption over deal. Found guilty of trying to solicit bribe from Thint, local subsidiary of French arms firm Thales, on behalf of Mr Zuma - then deputy president. Released on parole on health grounds after serving just over two years
Another official, Tony Yengeni, chairman of parliament's defence committee at time of deal and ANC chief whip, convicted of fraud in 2003. Also freed on parole after serving five months of four-year sentence
April 2016: commission of inquiry into deal found no further evidence of corruption or fraud
The opposition Democratic Alliance (DA) has waged a seven-year battle to get Mr Zuma to stand trial over the charges.
In a separate case in March, the Constitutional Court said Mr Zuma breached his oath of office by failing to repay government money used to upgrade his private residence with a swimming pool, amphitheatre, chicken run and cattle enclosure.
It backed an earlier ruling by an anti-corruption body that said $23m (£15m) of public money had been improperly spent on Mr Zuma's rural home in Nkandla in KwaZulu-Natal province.(BBC)