Laura,
Virus, adware, malware, etc. protection and insulation is the NUMBER ONE problem facing systems administrators today....period.
Would your server be vulnerable? Yes. Same as everyone else's is. The best you can do today to provide open and useable access to users while still maintaining reasonable security is to layer your protection.
Perimeter protection is your firewall to the server. Behind this you have your AV and spam protection. Behind that is your Active Directory (AD) policy control. Behind that is your adware, malware and related.
Beyond this, you HAVE to make sure you have appropriate use policies and that they are published, distributed, signed-off, and enforced.
By doing these things you will then be as safe as anyone else out there.
My 2 cents...