Well, I can see that, but I disagree. I think Narcissa has no intention of letting the ghost of Voldemort chase her out of her own home. She made it through that difficult time and now she wants to forget it ever happened and go back to a nice life with her son. She's quite annoyed at him for not going along with her plans.