This structure is a common use of the directional complement. It indicates the direction in which an action is being carried out. The direction is with regards to the speaker. 来 means the action is in the direction of the speaker。 去 means the action is directed away from the speaker.
Come down, let's go play football/soccer.
There is somebody outside the door, go out and take a look.