I've held the belief that Death Knights should not run off to Outland almost straight from their starting area. It seemed to make no sense. They break free of the Scourge and swear to destroy it, and yet right after they run off to Outland to fight an enemy which they may have never ever heard of. How could Illidan and Kael'thas possibly be a worthy distraction to their pursuit of Arthas? They have no vendetta with them.
I left out some factors. There are reasons for the Knights of the Ebon Blade to wish to venture into Outland.
The last thing you hear before leaving is the need for allies. The Alliance or the Horde is the ally. These allies seek to secure Outland. While it would not seem to be in the immediate interests of the DKs, to maintain ties it is worth supporting their allies.
While DKs leave their area with powerful armor, a potent runeblade, and the might of Death at their control, they are hardly in a position to directly challenge Arthas. They've obtained the most powerful armor of the Scourge, at least as they can tell, so it is unlikely that further battles in the Plaguelands will yield much gain. In contrast Outland is likely to have many undiscovered items of power.
Furthermore, Outland has entirely new challenges. It has direct confrontation with the Burning Legion in addition to all sorts of unknown enemies. To battle against them will give needed experience and versatility of tactics. It is a training ground for when they take the fight to Icecrown CItadel.
While I claimed that the DKs have no score to settle in Outland, this is not entirely true. They might know that the Scourge was created by the Burning Legion. While Arthas may have been their direct enemy, the Legion is responsible for the existence of the Scourge. Depending on the DK, they may even have been veterans of the previous wars against the Legion. An elf in particular could have been fighting them for thousands of years and could regard them as their main enemy even above Arthas.
Arthas would expect the DKs to chase right after him. He would be prepared for them. For them to go off to Outland instead would give the appearance that they are afraid. From that he might underestimate them. Then when they return, it is as a surprise, and with greater power for their struggles.
This was all general factors, things that affect all DKs. What about our own personal DKs? My orc, Weisserose went to rediscover his identity as an orc by facing the fel orcs and hoping to discover the untainted ones. My gnome, Fizzledbang, is drawn by rumors of arcane and demonic energy which he might be able to craft into a new and more powerful bomb.
Why is your Death Knight in Outland?