Prevent the contact creation while Lead conversion. Is this possible?

Yes,It's Possible.

 Description:

Create A checkbox Field in lead object (Created_By_Lead_Convert) and Select default checked .
Create A checkbox Field in Contact object (Created_By_Lead_Convert) and Select default unchecked.

Than go the Lead Object Fields And click on "  Map Lead Fields " button And map the Contact field with lead field.

Write A validation on contact object:

AND(
ISNEW(),
Created_By_Lead_Convert__c = TRUE,
CreatedBy.Profile.Name <> "System Administrator")