These are steps I went through to upgrade my databases after installing MySQL 5.5 from 5.1 on windows.
Presuming that c:\mysql is the installation path of MySQL on windows, at command prompt:
C:\> cd c:\mysql C:\MYSQL> bin\mysql_upgrade -p
Enter MySQL server password when it prompts:
Enter password:
That’s it.