To delete a database that has been set as Offline, you first have to bring it back online, and then drop it, like so:

EXEC rdsadmin.dbo.rds_set_database_online N'YOUR-DATABASE-HERE'
EXECUTE msdb.dbo.rds_drop_database N'YOUR-DATABASE-HERE'

If you have problems, you may try renaming the database and trying again:

EXEC rdsadmin.dbo.rds_modify_db_name N'YOUR-DATABASE-HERE', N'DeleteMe'

If none of these work, you may not have the correct permissions, which could require resetting your master password — see the AWS documentation for that.