Could I have something along the lines of social phobia?

Yeah, it sounds like you're a little social phobic. People generally are geared towards pride or shame in their interactions with others, so the fact that you're a little more afraid of people tends you towards the shame side. Fixing it is something that takes time and practice. The most effective method in my opinion is exposure. That is, putting yourself into situations you find uncomfortable and slowly getting used to them. Also, I would ask you why you don't feel like you're good enough for others. What is it about yourself that you feel good about? And what can you do to like yourself better? Maybe try getting good at something, music, etc. Any amount of confidence in your abilities builds your self-esteem and makes you less dependent on the opinions of others. Just remember that it will not be easy or fast, but it can be done.