im pretty sure in milkshape you select the face tool and then click on the 3 vertexes you want to create the face on, one at a time,
depending on the prder you click though you may make a backwards face or 2 in th eprocess,
though ive heard the newest versions have a "create face" tool that works how you say, in which you select the 3 vertexes and then click the "create face" button somewhere, i do not have the newest version so i cant confirm it, but seeing as you didnt mention the create face tool i think yourjust selecting the face tool instead of actually using it,
click on the 3 vertexes one at a time see if that works for you