There is no common law privilege against incriminating your spouse.
There never has been, it seems. Centuries ago in Britain a wife was neither competent or compellable to give evidence against her husband due to the legal fiction (derived from the Bible) that man and wife were the one flesh. Even by the time of the establishment of the Australian colonies that position was changing. David Lusty’s 2004 article, along with every other textbook dealing with the subject, will need to be revised.
But despite the media interest and some hysteria to the contrary, this won’t make the slightest difference to the operation of the law in practice. (Except, perhaps, where coercive investigative powers are in play). Section 18 of the Evidence Act (and before that, s 400 of the Crimes Act) provides a broad discretion to judicial officers to excuse people from giving evidence against their partners because of the potential damage the relationship may suffer. The UEA deals with this as an issue of compellability, not privilege.
(In NSW, s 19 creates an exception in the case of various violent and sexual offences. Those exceptions don’t exist here.)
This case, involving the Australian Crime Commission and its use of its inquisitorial powers, has been in the system for a while. It was discussed back here as Stoddart v Boulton  FCA 1108 back when it was an application for injunctive relief before the Federal Court. The decision was reversed by the Full Federal Court, then recently decided as Australian Crime Commission v Stoddart  HCA 47 before the High Court.
Statutory protection for spouses and significant others has existed for so long that it was reasonable to suppose the principle of spousal privilege, like so many others, has its foundation in judicial precedent. But [as Heydon J states unreservedly at 56] there is no case in the legal history books precisely on point, until the very recent decisions in coercive powers cases Callanan v B  QCA 478 and S v Boulton  FCAFC 99 which didn’t engage in a thorough historical analysis.
The appellant successfully urged the High Court not to, ‘join the dots and lend colour to an apparition that is really nothing more than an historical relic at best’.
French CJ and Gummow J [at 41, Crennan, Keiffel and Bell JJ in agreement]:
In our view, it cannot be said that at the time of the enactment of the Act in 2002 the common law in Australia recognised the privilege asserted by Mrs Stoddart or that it does so now. We agree with the conclusion of Kiefel J in Boulton  155 A Crim R 152 that in All Saints and the subsequent decisions, in particular Hoskyn and Riddle, the term “compellable” was used to indicate that the witness might be obliged to give evidence in the ordinary sense of the term, not that, in response to particular questions, a privilege might be claimed by the witness.
Heydon J dissented, embarking on his own detailed discussion of the legal history. Finding that not all principles of the common law can be found in ratio decedendi may be controversial to some (certainly the majority of the Court), but his finding that the ACC legislation does not contain the express language or necessary implication from Coco v The Queen (1994) 179 CLR 427 [at 438] is fairly orthodox reasoning.
The dissent is also worth reading for drawing attention to this little gem from Griffiths CJ in Riddle v The King  HCA 33; (1911) 12 CLR 622 [at 629], quoting an unnamed ‘distinguished lawyer from England’:
The law is always certain although no one may know what it is.