Note: In this article, ‘Window(s)’ refers to the window of an Application not the operating system. To refer to the operating system, I have written Microsoft Windows. ( in this article only )
Mac OS X functions on something I like to call ‘The Application System”.
The Application System. Think of it this way, if Mac functions on an Application system, then Windows functions on a window system.
A window is the visual area in an application containing some kind of user interface.
An Application or an ‘App’ is a computer software designed to help the user to perform specific tasks. An application may be capable of opening one or more windows.
For example, in your Microsoft Windows web browser, you may open one window for Google, one for Facebook and so on. Discard the idea of tabs at present and think of windows instead. Now, to close them, you would close each window one by one by clicking on the red cross in the corner. By clicking on this cross you are terminating the processes taking place in the windows. When the last window of the Web browser has been closed, the Application is no longer running.
This is not the case in Mac. In Mac, the window is a part of the application. So by clicking on the red cross in the top-right corner and closing the window, you are terminating processes within the window but not within the application.
So essentially, in Microsoft Windows, the window is both the window and the application. In Mac, the application contains the windows.
So in a Mac you could have no Application windows open, but the application may still be running, which, if done excessively, can make your computer slow. ( Don’t worry, you’ll get it as you use it )
So naturally, the obvious question arises: How do I close the Application and not the window in Mac ? When you close an Application, you are closing its windows as well. Closing an Application is called ‘Quitting’ the application. Shutting a Window, is called closing a window. 
To quit an application, click on the Application name in the menubar ( left ) and click the last option: Quit ApplicationName.
Or you could simply press Command-Q To close an Application window, select the window, click ‘File’ in the menubar and click ‘Close Window’ or ‘Close All Windows’
Alternatively, simply press Command-W ( Close Window ) or Command-Option-W ( Close All Windows )
Now, another obvious question that might arise is “How do I know which applications are open ?”
Apple has provided for this by making tiny blue indicators beneath open application icons in the Dock as shown above. Even if your application isn’t in the Dock it will appear in the Dock as the last icon if it is open. ( You can drag it from the its position as the last icon to any other place in the Dock to keep it there permanently or you can right click on the icon and click ‘Keep In Dock’ )
Alternatively, to view and switch between open applications, use the Application-Switcher by pressing Command-Tab. Keep Command pressed and pressed Tab repeatedly to switch between applications. Release both buttons to select the applicatiosn.
What you should take from this is - Use the application switcher from time to time and quit Applications that you’re not using.
Note: Finder is an Application that cannot be Quit.