WordPress core versions are released on a regular base, with security upgrades in between if needed.
You want to upgrade as soon as possible, and as of WordPress version 3.7 automatic background updates for security releases is introduced.
If you want more controle, or even get the newest Core, Plugins and Themes (aff) updated automatically? What is the easiest way to do that?
To help take the necessary steps for a secure upgrade, you can use a plugin called WordPress Automatic Updater.
This plugin upgrades your WordPress installation by doing the following:
1 .Backs up the files and provides a link to download.
2. Backs up the database and provides a link to download.
3. Downloads the latest files from http://wordpress.org/latest.zip and unzips file.
4. Puts the site in maintenance mode.
5. Deactivates all active plugins and remembers the settings.
6. Upgrades WordPress files.
7. Gives you a link that will open in a new window to upgrade installation.
8. Reactivates the plugins.
This plugin can also can be run in an automated mode where you don't have to click any links to go to the next step, but I prefer to have more control. It's as simple as clicking through to the next step and downloading the backup files (as the plugin suggests).
This website was upgraded to a new version in about 5 minutes. I did this without leaving the WordPress Dashboard or writing down which plugins were or were not active… and without doing any FTP uploads.
If you want minimal downtime when upgrading your website, I recommend using this plugin for a fast and easy upgrade to the latest WordPress version.