My skin starts itching when I get sweaty?

It's only my upper body for some reason and only when I wear certain shirts, and it's probably not a detergent issue since not all my shirts lead to itching. Even after I change these shirts though I still itch for a day or two afterwards when I sweat. Any ideas on a cause to this?