Nuzz. it all depends on the router but if you go into the routers configuration page (usually obtained through IP or the network interface) there should be an option somewhere.

if you still cant find it then google "Router+Model disable UPnP"