There is a difference between creating a Setting, and creating your imaginary world, aka World Building. Both are important.
World building is the big picture. You create the entire background for your story, much of which you may never actually use within your manuscript. You, as the author, need to know the details of your world. You need to know the limits of magic in your lands, where food or trade comes from, have some idea of the forms of government and the religions that your characters may live under. The less familiar your world is to the reader, the more you need to know. You have to be consistent. Once you set the rules, you must abide by them, or explain why not. World building is the universe within which your story unfolds.
Setting is just a fragment of that world. It is the small picture. Unless you are writing a short story, or telling the tale within a very limited scope, you will probably have to describe several settings. One way to illustrate the difference between world building and setting is to look at a story most people are familiar with…the sinking of the Titanic. If you begin your story in England or Ireland, perhaps even with the building of the ship, then travel across the Atlantic, perhaps to be rescued and end up in New York or Halifax, the world you are building is made up of all of those elements. A setting would be just one small piece or scene, such as the ship, or some action in the Grand Ballroom, or a romantic interlude in a drawing room.
While your world may just be a hazy part of the background of the tale as far as the reader is concerned, the setting can sometimes become almost like another character. The Titanic may be described in your story as a living entity, going through death-throes after the collision with the iceberg. Another example you may be familiar with is the old haunted house or the remote cabin in the woods. In a well-told tale, these settings can become as real, with feelings, as any of the characters who inhabit them.
The stronger you build your world, and the more time you take to create your settings, the greater impact your story will have. You, the author, are the stage manager behind the play your characters are putting on. While these elements should not dominate the plot, or turn your characters into cardboard cut-outs, they will enhance the enjoyment of what happens between the pages.
At least that is the way I see things.
The Dark Lady
The Queen’s Pawn
Dial M for Mudder
House on Hollow Hill