use libmagic to detect valid file formats

Writing our own magic bytes detection is prone to errors and an
everlasting catch-up-game. Let's use libmagic to get things right,
this is less code and makes things more reliable.

Building without libmagic is still possible. That will make the code
act like specifying FEH_SKIP_MAGIC=1, effectively passing everything
to imlib2.