To disable or prevent the directory access in your website if it does not contain any index file, add following in your .htaccess file.
Options -Indexes
# or #
IndexIgnore *
If user points the browsers to a directory which does not have index file, then a 403 error will be shown in the browser.

This file level permission will also be applicable to all sub folders that do not have any default index file.