With the current form of the game enemies go on a killing spree without valuing their lives. (With some small exceptions.) I think most enemies could actually be a bit stronger if they used caution? Like the mantis waiting for the ants to be few enough they can not do serious harm to him before attacking? Or tiger beetles running away when there are many ants nearby, denying the player its food value.?
This is already partly implemented with jumping spiders (although I expect you already knew this), since they'll run if they are suffering too much DPS. I just feel like pre-combat caution and saving their abilities fits the idea behind them and would be a good minor buff to an otherwise very weak enemy.
For other enemies, I feel like only some of them should have defensive and cautionary AI, as otherwise they would be far stronger than they need to be. The player should be able to hunt down a powerful enemy without it escaping. That said, I suspect some weaker enemies could do with better decision making, especially if they're smart.
Arachnids are currently unique in that every one of them has special AI quirks. Their behavior is not the standard enemy combat protocol, but involves special movement and target choice. This makes them a great target for behavioral advantages.
Wolf spiders outside of the nest are the best candidate for your suggestion, they could check the number of enemies in a wide area around a target to decide whether to back off or go for a kill, and use that strategy to pick off ants safely (sort of what jumping spiders would ideally do). To balance this out, they should not act this way in the nest, and they shouldn't flee from combat if they are repeatedly encountering a large group of opponents (so, if they recognize the same opponents are causing them to flee a few times, which might be caused by the ants remaining in a region for a long time or chasing down the wolf spider, it should just fight up-front). Alternatively, this sort of tactical hit-and-run could be saved for enemies added in later levels.
I am of the opinion that the self-inflicted fear on jumping spiders should eventually be replaced with more intelligent decision making, so that the jumping spider isn't merely forced to run away, but is intentionally losing aggro and repositioning to pick off targets. Given the complexity, this (and same for the suggestion above) will likely need to wait until the game is much further along, potentially even fully released.
AoE attacks in general could use more intelligent programming. Aiming for sufficiently large groups of enemies and holding off using those abilities when not battling more than a few ants (perhaps they still have a chance to use AoE, but it's a low probability) would make those abilities more dangerous. Despite making the game harder, this would remove much of the randomness of whether a rove A beetle is going to spray down 20 ants in-front of it or the two ants at the rear. M and L wolf spiders would also finally hit things with their venom, instead of using it immediately on the first two ants they encounter. The main reason L tiger beetles are so powerful is because they have a good chance of using their stun ability first, which means their AoE is delayed until most ants in melee are actually around them.
Enemies using DoT could also do with the AI to recognize there's a damaging debuff on an enemy, so they should have a higher chance attack foes which are not suffering damage debuffs first so as not to waste hits.