Is it possible to take panic attacks due to the presence of a particular person?

Yes. It is absolutely possible to have panic attacks in the presence of a particular person, or even a particular type of person. You did not mention if you were in therapy or counseling for these panic attacks, but it may be a good idea to start there. Confronting someone when you are in a vulnerable state is rarely a good idea. It opens up the possibility of the person sensing your vulnerability and using that opportunity to take power over you again. Please see a professional about this and decide, in a safe environment, whether you are healthy enough to confront this person. That is not always necessary for recovery, either. Many people are able to overcome panic attacks without having a confrontation. In the meantime, make a plan with yourself as to how you are going to handle the situation if it comes up again. For instance, plan that if you run into this person, you will casually veer off in another direction. Or, plan to say hello and then move away from the person. Make sure that you are choosing something that feels healthy, not something that will make you feel vulnerable. Then, try to put it out of your mind. Having a plan restores your power, because really, the only person you can fix is yourself. Unfortunately, nothing you can do will change someone else. Therefore, take as much control of yourself as possible, and give yourself credit for having done something healthy.