I'm not sure if this would work for the Basilisk or the Carnelian, each of which starts with no weapons and thus an inability to take down shields.... Every ship (with the exception of some automated drones) has at least 1 shield.Don't allow boarding crews to board until the shields are down
I agree that boarding is somewhat OP, but it does come with a fair share of risks... I'm not sure if there's an easy way to remedy this. The teleporter lock is a pretty good idea, making boarding even more risky. But, again, for ships that depend on boarding in the early stages if the RNG on teleporter locks screws you over you can easily lose all your guys if you need to get them out ASAP (i.e. an automated drone) .
Maybe make melee attacks less effective while boarding? Give some ships (i.e. those with only two crew, both engie or zoltan) some sort of counter to a couple yellow mantis' -- like doors that work. Enemy ships, IIRC, don't have airlocks -- but, if they did, and the AI had the power to use them and the enemy crew actually ran away instead of fighting a losing battle -- this would definitely make things more interesting.