Dementors affected Harry Potter the most, and a fan theory suggests it was not just because he had a tragic childhood and lots of bad memories.