Small improvement for the dkbhadeshiya's answer: you don't have to do cd dir, just specify working directory to the tar instead:
ssh user@host "tar -C /path/to/data/ -zc directory_name" | tar zx
You can also upload directory the same way:
tar zc directory_name/ | ssh user@host "tar zx -C /new/path/to/data/"