Configure scale, height and brightness of emotes in the config screen (Mods -> Pop-up Emotes -> Config)
How to replace default emotes with a texture pack: Get resource pack template here and if your texture pack has less emotes than the default emount, hide what's left by specifying resourcePackEmoteAmount in the client config
You can disallow custom emotes in the server config
Version 1.3.6 and lower:
Add emotes by placing .png files in the pop_emote_pack folder inside your Minecraft instance
Players will be able to see each other's custom emotes after the game server synchronizes them