If you are buying fresh swordfish, buy the freshest you can. Like any other fresh fish, it should not smell “fishy” at all. Really, at all. Do not buy fresh fish to eat tomorrow. Cook it and eat it the day you buy it. Looking at the swordfish steaks in the market, any coloration should