The nightmare is a large, black, wyvern-like creature with a large column of teeth and large horizontal wings. It drops the Nightmare scale which is one of the ingredients to craft the Big Bertha. It also drops rotten flesh and the zookeeper shard.
Spawning
The nightmare spawns in the unstable dimension which is the dimension where you find floating islands. It is not particularly rare if you can fly above tree level. If you want to fight it easier, you can spawn it from the ancient dried spawn egg.
Combat
The Nightmare is a difficult boss as it can vary in sizes and damage. It is stronger than most bosses but weaker than Mobzilla, and the Nightmare is the only mob to actually hurt mobzilla. The minimum and maximum damage it does is 20 - 160, 9 - 30 for defense, and 75 - 1000 health. You can tell how much health a nightmare has because of it's size. The smallest nightmare is about 2 blocks long. It is recommended to have the full ultimate set as the Nightmare deals a lot of damage. It drops Zookeeper shard('s), feather('s), item frame('s), raw beef, rotten flesh and nightmare scale(s).
Trivia
The Nightmare is based of the creatures in the movie Pitch black.