You could have 1000 upload and 1000 download and still have a poor connection.
Download speeds mean SQUAT in online gaming. What needs to be strong is your upload, ping, good jitter and zero packet loss.
There are TONS of factors that play into this. The line quality to your house, the line quality in your house, the quality of your router, the ISP itself my limit traffic flow at certain times, your cables from your router to XBOX, if you are using wireless there may be interference, the settings on your router, and so on.
While in an ideal world I would like to see a minimum requirement to be allowed to play competitive style games online, it will never happen. If your ping is too high, or you have a lot of jitter/packet loss, it should kick you out until those problems are resolved.