Although your clothes weigh you down somewhat, the first few strokes of your swimming technique easily parts the water and takes you closer toward the other island.
Without warning though, a rip-current surges beneath you a few further strokes in to your swim. As much as you try to swim against the current, it is useless.
You have two broad choices:
Swim with the current, allowing it to take you where it will.
Swim tangentially to the current, heading back toward the beach with the teleporter pad.