It doesn't matter, not legally anyway except in extreme circumstances. But that doesn't mean of course that it does not matter. If it matters to you, a lot, that you divorce your spouse then it is an important issue for you and to do so may help your healing process.
But let's be clear. These days the Courts are not really interested in who did what to whom. Their attitude is they have heard it all before and if both want a divorce they may as well have one.
A common misconception is that a divorce on the grounds of adultery or unreasonable behaviour will affect the divorce settlement. It won't unless the behaviour is extremely bad (and for a divorce Court to think it is bad, it has to be really bad) or is really bad financial misbehaviour. Adultery is not relevant at all, although living with that person is, as you'll no doubt be partly pooling resources.
There is more info on divorce grounds here.