Speculators are leeches. They buy low, sell high, and contribute nothing. They're a symptom of a poorly functioning economy, an economy filled with ignorant and lazy participants. In other words, they're your fault.
Sometimes infinite dust is around 1g each. Sometimes it's closer to 2g each. There's money to be gotten by buying it at 1g and selling it at 2g. I do this when I'm tired of all the thinking involved in being productive. I could get a greater profit by making and selling scrolls, but I'd have to see which scrolls are up, track mat costs, figure out demand, etc. Speculation is lazy gold.
It's not entirely worthless as an activity, despite my hyperbole at the start. Buying when there is excess supply helps keep up prices, supporting suppliers when they would otherwise be having trouble selling. Conversely, selling when there is a short supply helps keep down prices, supporting buyers (who are probably crafters, and therefore the productive ones) when they'd otherwise be unable to find profitable crafts.
But the speculator is unnecessary. He does nothing that others cannot. Instead he has a small piece of knowledge which is very valuable: You do not have to sell anything at a given time, nor do you have to buy. Gatherers and crafters could use this too and they'd be richer for it.
You've just farmed ore for two hours and are headed to the AH. Check the prices and... why is it half what it is normally? You'd better sell twice as much, just to break even! Or, don't sell any. Don't add your ore to the obvious over-supply. Hold on to it. Wait a few hours or a few days. Surely you can survive that long. You can keep farming, just don't sell it yet. Let the crafters buy it up. The extra gold you'll get from this would have otherwise gone to a speculator, or a crafter who got a really great deal, either way, you have more gold.
Why is the AH flooded with belt buckles? You can't sell at that price, you'd be at a loss! So don't. Keep crafting if you want, but don't sell. Check if material prices have crashed and that's why the crafted price is so low. You don't have to be part of the problem, in fact you can make more gold by being part of the solution.
You've probably figured out that all I'm really saying is to self-speculate, to "hold low, sell high". In effect, there are still speculators, they're just using their own materials. But there would be no speculator 'class', meaning they will no longer be able to make preachy bossy self-promoting economics posts about how they're brilliant.