A traditional romantic novel always has a happy ending: boy gets girl. A good science-fiction story must contain elements of understandable science: it must obey some rules, even if they are strained to the limits of the author’s imagination. Within these broad genres, there are many, many sub-genres: under ‘mystery’ or ‘crime’ you find the hard-boiled private-eye stories of Mickey Spillane, the amateur sleuths of Agatha Christie or Simon Brett, and the insider’s-look-at-the-forensics of Patricia Cornwell or Kathy Reichs.
If you are writing genre fiction it is important to understand these niches and therefore to understand your market. Is your mystery novel a police procedural like Ed McBain’s or a techno-thriller à la Tom Clancy? Is your science-fiction story psychological or does it depend on your characters meeting weird and wonderful beings from another world? Or both? A good look at the market and plenty of objective homework categorizing your novel will be time well spent.
Some writers are rebels, defying classification for their work. Are Anne Rice’s vampire novels horror or romance? Readers have their own opinions. It’s not against the rules to write a book that contains elements of more than one genre, as long as you understand the combination and have given some thought to the market.