If they are swimming off the bottom to get the bait these not much which can be done. But adding a float or three to your dropper can help get the bait further off the bottom which might slow them down.
If you use tougher baits such as a Kahawai fillet folded in half so the skin is on the outside, then wrap it shut with heaps of bait elastic can slow them down. Another option is to start using them for bait.
Can use cotton mesh bags, but crabs would rip them apart to.