The Library Contents section gives you an overview of what is available in your scene, in the user libraries and in the standard libraries.
The section is divided in several 'tabs' for Objects, Fixtures, Materials and Accessories.
Objects
Fixtures
Materials
Accessories
For the first three, you can simply drag-and-drop an item from the list into the scene (in case of an material, onto an object).
If the selected item was not yet part of the scene, it will be added to the inventory of the scene as well.
For more information on object/fixture insertion and placement, please see the section Creating a Scene.