Books with this trope: protagonist haunted/followed throughout story by an incorporeal entity (who is usually either malevolent or at least a trickster)

Some examples first because what I'm lookin for is kinda specific (SPOILERS for Next to Normal, Mistborn, and The Invisible Life of Addie Larue):

1. In Brian Yorkey's musical Next to Normal, a mentally ill mother hallucinates her dead son throughout the story and he acts as a sort of Stephen King-like entity, being the story's catch-all manifestation of grief/mental illness (the way Pennywise is this for the concept of fear). He's not ACTUALLY an independent character but more metaphorical, making this a kinda psychological ghost story, without the supernatural element. He is both an antagonistic shadow looming over the story while also still having friendly and emotional interactions with the mother, as her delusions (while being harmful) also stem from a place of pure maternal love.

2. In Mistborn, Vin hallucinates her abusive brother, and his voice tends to echo in her head in moments of self doubt, mocking or goading her. His shadow over her looms strong, and is the main reason she has such a hard time making friends. It's eventually revealed that Ruin, the main antagonist (an ancient dark god) has been responsible for her hallucinations and took form of her brother since it'd be the most effective way to get to her. Even after this reveal, Ruin kinda just hangs around Vin at times, watching with amusement as she goes around, trying to gather allies and foil him. He just pops up at times and comments/annoys, in a way that almost reminds me of Beetlejuice? A dark entity that absolutely has dark designs down the road but has no problem with kinda just chatting it up with the protag.

3. In the Invisible Life of Addie Larue, our main character makes a deal with the devil by wishing to live forever with the caveat that if she ever wants to stop living, she has to relinquish her soul to him. Thus ensues a decades long game between the two where she desperately finds reasons to go on in an ever changing and chaotic world while he pops in to check on her with nothing substantial to say half the time aside from teasing and mocking, hoping to eventually break her will, even if he has to resort to some cheating down the road. I actually really love their dynamic: he's a complete and utter monster and the story doesn't deny this but he has just enough human characteristics to charm both her and the reader. And you never know how much of what the actual Father of Lies is saying is even authentic to himself or if he dons a different persona to suit the person he's seducing/corrupting.

So to sum it up: not looking for ACTUAL traditional horror stories with ghost hauntings so much as this dynamic, in whatever genre it might take form. The entity tends to embody a deeply personal flaw or trauma our protagonist struggles with, and is antagonistic WITHOUT being an utterly alien and incomprehensible evil like Cthulhu or Sauron. Yes they can be of a similar nature as those characters in-universe but their roles are different: we never get Sauron having moments of gloating or mockery or Cthulhu tapping into a protagonists personal fears with visible glee while exhibiting a distinct personality. Sauron and Cthulhu are almost non-entities and are still great at causing fear of the unknown. I'm looking for entities that are UNCOMFORTABLY known and familiar to the protagonist, but are no less a threat or menace by the end.