Now I know there are exceptions of this rule. One is when the narrative style is set up from the start to be a kind of internal monologue by a character. This character does not have to be one in the story; the narrator, even a detached third-person one, can be a kind of character for the telling of the story. Even Jonathan Swift, that proponent of a strict 'language police', has done this and with good effect.
Another exception, of course, is when the narrator is an active character in the book, such as in my own Deirdre series and in countless other works. In such examples the narrator's discussion style can either make or break the whole story. (I will boast that the one consistent observation of especially female readers is about how 'right' I seem to have got the heroine's thought processes. So, it works.) But a narrator with charisma and, even more important, something of merit to say, will win over the readership every time.
A very personable narrator, even a detached one, can be effected by the use of narrative and even grammatical devices. In Deirdre the first-person narrator is essentially articulate and perhaps of above-average education for her age, and she does use some fragments but certainly not to any excess. The story is about events and information, not about her personal reactions to what happens (those come through in her actions and words themselves) and so frags like 'Maybe not.' and 'In my face.' do not occur as a rule.
In dialogue the matter is quite different. Few people speak habitually in absolutely-correct SWEE (Standard Written and Edited English) and the use of fragments, unfinished thoughts, interjections such as 'um' and 'uh', verbal quirks and other shortcomings in usage and grammar can be very deftly used to reinforce the uniqueness of the characters. But the rule here must be one of consistency. Many writers make the error of making all their characters talk the same way, with the same (the author's, usually) idiom, grammar and vocabulary. Keeping character's speech patterns distinctive can also help in eliminating the too frequent repetitions of 'he said' and 'she said', as the separate speaking styles of the characters will differentiate them from each other.
Sadly, as with dialogue, it is all too easy for an author to fall into his own verbal style when writing the narrative. It's too easily overlooked or taken for granted; and the all-too-common post-romantic self-focus deludes many writers into believing that how they use the language as individuals is how best to communicate a novel's story to the world at large. This is akin to constructing a house with poor foundation and framework-- no matter how prettily you dress it up inside and out, and no matter how workable the form and function of the design, no-one will be able to long ignore the inadequacies of the fundamental structure because they will affect every other part of the visit.
I believe the story should have its own 'character', sort of like I said above. The whole 'feel' and 'mood' of a novel comes across by how the narrative is presented. For my part I choose to hold the narrative itself somewhat elevated from the dialogue, insisting on SWEE as well as it can be done. One of my projects (still long in the future) is a series of 1740s novels of manners in which the narrative is given in 20th/21st-C English, to ensure accessibility to modern readers, but lines of dialogue appear in the mid-18th-C English of Johnson, both to be historically accurate and to illuminate readers as to how people really talked then. This is almost the opposite of what I did in a book like Love Me Do, in which the narrative is in the same reliable Queen's English but the dialogue appears very much in 1970s middle-class American vernacular.
Of course there are as many approaches to narrative as there are authors. For what I write, I do not have value for too conversational a style. I view a novel as much more sacrosanct than that; and, for most novel-length endeavours, writing like a comprehensive-school eighth-year with slang, jargon, abbreviations and numerals, phonetic spelling, grammatical imperfections and sentence fragments will not do. This is not just an academic point to be read and disregarded over time. I would make the entirely unequivocal submission that in such inadequate writing the whole point of the story itself can become lost. And, if anything, the prime directive of all writing should be to not irritate and disinterest an otherwise eager reader through an author's misuse of the language. After all, an author is writing to be read and to be published; and to have his work to die a premature death due to poor editing and preparation with regard to basic grammar and usage has got to be the worst way to go. --and the most easily avoided.
* * *