The answer to your question is not so black and white as some people may think. Let me attempt to explain my view as follows:
1 ) In regard to the principles of Islam that are self-evident in the sources, no one is required to follow anyone but the Law-giver Himself; for example, this is the case on issues such as the obligation to pray, give zakah, fast, make the pilgrimage. Likewise, we do not need an expert to tell us about the heinous nature of the cardinal sins clearly forbidden in the sources.
2 ) The question of following a madhhab (school of jurisprudence) comes only in regard to issues of fiqh where things are not self-evident in the sources. Therefore, one needs to rely on the experts. In this case, again, the issue is not black and white --it all depends about whom one is referring to.
3 ) It is said,' a common man has no madhhab', for his madhab is that of his teacher or mufti. For what is required of him to do is to rely on those who are more knowledgeable than him. Since he possesses no knowledge to base such a decision on, it would be wrong for him to say I am a hanafi or shafi'i or hanbali or maliki. To do so is not different from someone saying I am a writer, I am a doctor, etc., without any knowledge in such areas.
4 ) But, I must qualify what I said above by stating what some scholars like Imam Shah Waliullah said: Since Islam reached the people in various parts of the world through scholars, following various schools, people conveniently identify themselves as hanafites, shafi'ites, malikites, etc. For instance, people in South India (especially Kerala) generally consider themselves as shafi'ites, even as those from North India often prefer to call themselves as hanafites, albeit there are still exceptions to this rule.
5 ) Madhab, however, effectively comes into the picture when a person is embarking on the study of fiqh. Since no one starts with comparative jurisprudence, they start the study with a text from a specific school. It is, therefore, inconceivable at this stage for anyone to choose another school, of which he has hardly any knowledge.
6 ) A person who is thus exposed only to a single school cannot be considered an expert in fiqh. The adage goes,' whoever is not aware of divergences of views in regard to fiqh, they did not even smell fiqh`. In other words, to know fiqh is to know the differences of jurists, along with the evidence.
7 ) Once a person has advanced to this level of knowledge, where he is exposed to comparative jurisprudence and thus is aware of differences of opinion, he is not bound to follow a single madhhab in every single issue he is faced with. He may choose the views of authorities or jurists whose rulings are the strongest, or more relevant to a specific situation, or more understandable to him, as he is not expected to recommend an opinion, if he is not quite sure of its rationale. Scholars have said, `No one is allowed to give fatwa by our ruling, unless he is sure of our evidence'.
8 ) Having said this, I must still remove a misconception. A person who does so must still follow the authorities, for knowledge must always be based on sound methodology. Thus a person who is aware of the divergences of views would still be following the authorities as he cannot follow his own whims; he can only offer advice or rulings following the acceptable methodology and rules of fatwa.
9 ) Lastly, if a scholar were to simply parrot out the rulings as compiled in the books, on the pretext that he must follow a single madhhab in all cases, without regard to the milieu, and the specific circumstances of people, he ends up distorting rather than serving the shari'a. I must end this by citing an illuminative quote from Imam Ibn al-Qayyim:
"Whoever issues rulings to the people merely on the basis of what is transmitted in the compendia despite differences in their customs, usages, times, conditions and the special circumstances of their situations has gone astray and leads others astray. His crime against the religion is greater than the crime of a physician who gives people medical prescriptions without regard to the differences of their climes, norms, the times they live in, and their physical conditions but merely in accordance with what he finds written down in some medical book about people with similar anatomies. Such is an ignorant physician; the other is an ignorant jurisconsult but more detrimental" .