There are so many different types of yoga and different teaching styles. I strongly suggest you try a variety of classes and a variety of teachers before you give up on yoga. Look for a beginner or introductory classes, or try private classes.
Yoga, Meditation, Wellness