No. A doctor typically cannot withhold your medical records because under HIPAA a patient is entitled to access to their medical records. The only instance when a doctor can refuse to release medical records is if they believe they might be harmful to the patient. This includes physician notes regarding mental health or other records like HIV tests that can be regulated by state-specific or federal laws.