I assume that the device name of your external hard drive is "/dev/sdb". You should replace all "sdb"s below, if your device name is different.
Step 1: Wipe out all your data on your disk:
sudo dd if=/dev/zero of=/dev/sdb bs=4K
This step might take some time (*).
Step 2: Create a partition, which you want to encrypt, on your drive. You can create more partitions, of course. You can use command line tools such as "fdisk" but I recommend the "partitionmanager" of KDE, or "gparted" of GNOME.
After you create the partition of interest, note down its name. In my case, it is "/dev/sdb1". If you created a on big primary partition in your external HDD, then it is most likely to be names as "/dev/sdb1".
Step 3: Setup encryption.
sudo cryptsetup --verbose --verify-passphrase luksFormat /dev/sdb1
You will be asked a password here. Choose a strong one and don't forget this password. You will use it whenever you want to mount your device again.
Step 4: Open the encrypted partition for use:
sudo cryptsetup luksOpen /dev/sdb1 externalHDD
You can replace "externalHDD" with whatever name you like. This command will create "/dev/mapper/externalHDD".
Step 5: Create a filesystem on the encrypted partition. I chose ext4 because it is the default being used by the latest Ubuntu variants:
sudo mkfs.ext4 /dev/mapper/externalHDD
Step 6: Now your encrypted partition is ready to use. You can mount the drive using command line:
sudo mount /dev/mapper/externalHDD /mnt/extHDD
or, you can use the Device Notifier/Manager of your desktop. When I plugin my external HDD, I get the following notification in the lower right corner of my screen:
Notice the little locker icon on the right hand side. Click on it, and then you will be asked the password you set in Step 3 above:
If you enter the correct password, then you will be able to access the encrypted files on your external HDD:
When you are finished using the HDD and you want to unmount it, simply click the eject button in the Device Notifier. The eject button is the circular button with a triangle and a line inside it, in the above image.
I have used the following resources to prepare this post:
(*) It actually took me about 8 hours for my USB 2.0 1TB external HDD. I really don't know why this step is necessary. Source  above says that "... writing data to the partition prior to encryption helps protect against data attacks, finding patterns on the block-level, etc. " . Please comment below if you have any knowledge about this.