What would you say is causing this...

[Timmah!]

This is the only tomato damaged on the plant & it's only on the shaded side of the fruit. The other side is blemish free. The spots are superficial and can be scratched off with a fingernail.

[puzzley]

I'm certainly not sure, but it looks like feeding damage to me. Don't know what, but some kind of chewing insects Any potato beetles around? They will feed on tomatoes, and I've had a few this year....

[RandyG]

It's bacterial spot. Probably the reason the shaded side is infected is that it stays wet longer than the non-shaded side and the free water favors the bacterial disease development. You can find lots of pictures showing bacterial fruit spot symptoms similar to this on the internet.

[ssi912]

looks like juvenile army worm damage. the small army worms will munch on the flower bracts in my experience. but you say you can wipe the spots off? dont know what would cause that. spray some insecticide that will kill worms and such. process of elimination. the juvenile worms are really hard to see. almost translucent. good luck.

[Mischka]

I think that RandyG is correct; that is bacterial spot (not speck). I've seen it here after several days of rain, especially in years where I've spaced plant rows closely together.

Further evidence to support this is that you're able to wipe off the spots. Army worms pierce the dermis and cause damage to the underlying tissue and it obviously couldn't be wiped off.

[RandyG]

There are three different bacterial diseases that result in fruit lesions. Bacterial spot results in irregular shaped scabby brown lesions on the fruit that are raised above the fruit surface. Bacterial canker lesions are slightly raised and have a dark brown, small, round lesion in the center of the spot surrounded by a white halo resulting in what is termed a bird's eye spot. Bacterial speck has a slightly indented , small black pinpoint lesion with no halo around the lesion. Symptoms on foliage, fruit pedicels and the peduncle for the thee diseases can be somewhat similar so the easiest diagnosis of which bacterial disease is present is based on the differences in the fruit spots. These bacterial diseases, especially canker and spot, can be very devastating. Also, chemicals for effective control of bacterial diseases are limited and often not very effective. All of these diseases can be seed borne so having clean seed and following sanitation practices to prevent spread are important.