Yeah scaring people requires the proper atmosphere, setting and leaving a lot to the audiences imagination. Unknown sounds, from Unknown things and the like.
One of the scariest scenes I have ever saw was a scene from Jurassic Park. The one with the Heroine in the Maintenance building trying to turn the fences back on. The arm of one of the park employees taps her shoulder but when she turns to look she notices the arm's all alone. Everyone in the theater jumped.