Enemies will creep along the path to the cave, steal one gem, then carry it back. Prevent them from stealing all the gems by building dens, crypts, and temples that will automatically attack them as they pass by.