ESPN has done a research into various sports to test how difficult it is to play. The table tennis game is listed 47 in the list compared to Boxing which is listed as the hardest sport to play. You can find the complete rating list here. In Hand-Eye cordination, table tennis is second next to Baseball/softball - I would consider the game Cricket higher than baseball.
ENDURANCE: The ability to continue to perform a skill or action for long periods of time. Example: Lance Armstrong
STRENGTH: The ability to produce force. Example: NFL linebackers.
POWER: The ability to produce strength in the shortest possible time. Example: Barry Bonds.
SPEED: The ability to move quickly. Example: Marion Jones, Maurice Green.
AGILITY: The ability to change direction quickly. Example: Derek Jeter, Mia Hamm.
FLEXIBILITY: The ability to stretch the joints across a large range of motion. Example: Gymnasts, divers.
NERVE: The ability to overcome fear. Example: High-board divers, race-car drivers, ski jumpers.
DURABILITY: The ability to withstand physical punishment over a long period of time. Example: NBA/NHL players.
HAND-EYE COORDINATION: The ability to react quickly to sensory perception. Example: A hitter reacting to a breaking pitch; a drag racer timing acceleration to the green light.
ANALYTIC APTITUDE: The ability to evaluate and react appropriately to strategic situations. Example: Joe Montana reading a defense; basketball point guard on a fast break.
SPORT END STR PWR SPD AGI FLX NER DUR HAN ANA TOTAL RANK
Boxing 8.63 8.13 8.63 6.38 6.25 4.38 8.88 8.50 7.00 5.63 72 1
Table Tennis 3.50 2.50 4.63 4.13 5.88 4.25 1.38 1.88 8.88 6.00 43 47