Judge Denny Chin postponed a fairness hearing set for next month that was meant to address a settlement between Google and authors and publishers.
The $125m agreement, worked out last year, has effectively been sent back to the drawing board by the judge.
The class action case would let Google distribute and sell digital versions of out of print, copyrighted books.
It has been criticised because some say it would give Google too much power to set book prices.
Judge Chin's decision comes after the American Association of Publishers and the Authors Guild asked the court to delay the final fairness hearing on the proposed agreement, which needs court approval to go ahead.
The $125m agreement, worked out last year, has effectively been sent back to the drawing board by the judge.
The class action case would let Google distribute and sell digital versions of out of print, copyrighted books.
It has been criticised because some say it would give Google too much power to set book prices.
Judge Chin's decision comes after the American Association of Publishers and the Authors Guild asked the court to delay the final fairness hearing on the proposed agreement, which needs court approval to go ahead.
Fair concerns'
The move follows objections filed by the Department of Justice last week. It said the deal should not go through in its current form.
In a 32-page filing, the DOJ said the settlement needed to be reworked so that it complied with US copyright and antitrust laws.
It was revealed earlier this week that publishers, authors and Google have been working to modify the agreement, which was completed last year.
"Under all the circumstances, it makes no sense to conduct a hearing on the fairness and reasonableness of the current settlement agreement, as it does not appear that the current settlement will be the operative one," Judge Chin wrote.
He noted that objectors to the settlement include "countries, states, non-profit organisations, and prominent authors and law professors".
"Clearly, fair concerns have been raised. It would appear that if a fair and reasonable settlement can be struck, the public would benefit," said Judge Chin.
'Status conference'
In 2001, book authors and the Authors Guild filed a class-action lawsuit, as did publishers, alleging that Google had violated copyright laws by scanning books from the libraries of major universities without always getting permission from the copyright owners of the books.
Google claimed at the time it was protected by the "fair use" principle because its book search engine showed only short snippets of text for the books it had scanned without permission.
Microsoft, Amazon and Yahoo have filed objections to the settlement with the court, along with the French and German governments, privacy advocates and consumer watchdog groups.
"Clearly voices such as ours had an impact on Judge Chin," wrote consumer watchdog advocate John Simpson in an email to BBC News.
"There was no way the proposed settlement could go forward. We believe that the proper place to solve many of the case's thorniest problems, such as that of orphan books, is in Congress because it is important to build digital libraries."
Orphan books - of which there are thought to be five million - are titles where the authors cannot be found.
Judge Chin has called for a "status conference" to be held on 7 October - the original date for the hearing - to determine "how to proceed with the case as expeditiously as possible".