Well a 240mm rad approximately has the ability to dissipate 450Watts.
A stock GTX 590 at full load would get you 330Watts according to reviews, when overclocked it could reach up to 400 Watts.
So the radiator definitely is not the problem, at least from a performance point of view.
This loop is fairly small, so even if the D5 would impose a problem I do not think it would be noticable in such a small loop, so letīs cancel that too, but to be safe check if the flow is reasonable so you can positively strike out the pump.
My guess is the GPU block.
Did you assemble it yourself?
I would disassemble the GPU block, check it, pads etc and reassemble.
Also make sure you have the latest drivers installed, and check if those insane temps also appear with other games, benchmarks and stess tests.