Recognizing a threat requires every sense you have.
While it's not hard to spot the author standing behind the light pole in this photo, you might easily miss him in real life. Learn to "see" rather than just "look" and develop your other senses to warn you of potential threats.
The first rule of self defense is being able to recognize a threat, and you can't do that unless you are paying full attention to the world around you. How does one accomplish this? By becoming truly aware of your environment.
Being aware is knowing who and what is in your world at all times. There's no need to keep reading this if you habitually surround yourself with input-distraction-devices--such as iPod ear buds or BlueTooth wireless phone connections--and are not willing to relegate their use only to times when you deliberately want to block out the world. From my knowledge and observations, the aural input strongly overrides your other senses.
To be aware of who and what is in your world, you have to look, listen and even feel when in any environment. First, you need to realize that "looking" and "seeing" are not synonymous. Learning to observe and continually practicing seeing has now become an acquired skill for humans.
A method I've used to teach seeing is to, while having a conversation with a student, ask him about what he observes in our immediate area. (The preconditions, to be fair here, are that he is not encumbered by connections to any electronic devices.)
Generally, the student is able to, without looking, tell me the presence of others but is not able to further describe them. He also can indicate where buildings stand, but without further details. The students do learn to look around, albeit in an awkward manner, as they anticipate me asking again later.
If possible, I then drive each of them around at a leisurely pace while asking similar questions: Did they see anyone on the sidewalk? What were they wearing? What were they doing? What stores or buildings did we pass?
If I don't ask these questions earlier, before driving around, almost invariably their first response is, "I didn't see anyone." Fortunately, most everyone picks up quickly on this game.
They do get a bit frustrated, though, when the game is changed to hearing and feeling. Unfortunately, the cacophony of noise in society is such that we all have learned to block out much of the background "noise" and have done it so well that we never turn it back on.
Given the space allotted here, I can only state the one truth of sounds: A threat will almost never make the same sound twice. For example, in your home an invader will not step on the same squeaky toy or stair twice. If you hear it, believe it.
Feeling is a very difficult sense to tune up, particularly when living and working in crowded environments. This is why pickpocket teams are so successful. One bumps ("stalls") you, and a second takes your wallet ("dips") and passes it off to a third (and perhaps even a fourth) person. A simple self-test to determine if your "feel" indicator is up and alert is to try discerning the wind direction at any given time.
One last sense that strongly goes with defensive tactics is your ability to smell. People have odors, and if you haven't seen them or felt them through contact or air currents, you can often smell them. For example, fear has a smell. In searching houses for fugitives, I often could smell their fear, which, for me, has the smell (and taste) of brass. Along with this smell of fear often comes stomach and intestinal tract disruption, so you get burps, stomachs growling and methane gas.
Also, the various colognes, perfumes and deodorants worn are quite noticeable, particularly when the wearers are using them to cover up a lack of personal hygiene. Smokers leave odor trails and wear the odor of burned tobacco.
At this point, it is reasonable to ask, "Who wants to live like this?" Well, the predators do, and wishing you didn't live in the same world will not make it so. The solution is learning these defensive tactics and practicing them till they become habitual.
Of course, these tactics do have to replace existing habits, many of which run counter to good defensive tactics (like walking around tuned in to various wireless devices, as I cautioned against earlier).
The learning curve isn't all that steep, though, and as you redevelop these sensors, you can then expand your ability to gather information under more difficult conditions, such as in darkness and in crowds like those found in airports and shopping malls. In such places, you are forced to make micro-second evaluations, discarding the unimportant lest you lock up due to information overload.
I'll leave you with this thought, a truism uttered in the 1998 movie "Ronin" by its protagonist, Robert De Niro, playing a former CIA field officer: "Whenever there is any doubt, there is no doubt."