Drupal's HTML filtering is an important security feature - we wouldn't want any blogger to be able to post JavaScript tags because that's how XSS attacks - or worse - are launched. In Drupal, unlike other blog systems like WordPress, you can't assume that the people who are allowed to create content are trusted. On many Drupal sites anyone can sign up for an account and start blogging. If those sites allowed JavaScript tags or even form tags to get through the filters it would quickly become ripe with bots and bad people doing naughty things.
That's why Drupal has different Input formats. For our untrusted bloggers and people who are allowed to post comments on those blogs, we need to make sure those tags are locked down to keep the site secure (hence filtered HTML) but for our trusted users we can allow more freedom. I usually recommend adding an additional input format for the average trusted user that is somewhere in-between Filtered (which is the default for comments, so you don't want to change that one) and Full (which has absolutely no protection, and should really only be used by people who actually understand, and want to enter FULL HTML).
To prevent the ugly "Text-formats" select widget from appearing, try the better formats module. This module allows you to hide the input formats from your posters via permissions. (Be careful if you are using the WYSIWYG module, you may need to apply a patch to allow WYSIWYG to work with better formats and also a patch to clean up the unnecessary white-space left behind.)