USB ports are an absolutely essential element in any PC, so you can connect all manner of USB devices to it.
However, if you are really worried about the use of these devices, then you need a tool like Good USB Lock. It is a utility that lets you block USB ports, so no-one can use them unless they have the password. That means they can`t make copies of anything stored on your PC.
Despite the name of the program, Good USB Lock, it isn`t limited to blocking/unblocking the USB ports, as it can also do this to the CD/DVD drives too.