Here's my mediocre kindergarten tier sketch of the town layout. walled and offset from the water (I assume we don't want the fishery and quay to be landlocked).
If you think you can do better, well you are probably right, and I more than welcome you to go ahead so I can delete this abomination.