I am reading THE PERSIAN BOY by Mary Renault. She describes how young boys were castrated to be kept as sex slaves for the aristocracy. They were not a part of the harem, but served in a parallel fashion.