The Latin abbreviations "i.e." and "e.g." come up very frequently in writing and would probably come up more often if people were more sure of when it is right to use "i.e." and when "e.g." is required. To me, the only way to figure it out is to know what they stand for.
"I.e." stands simply for "that is," which written out fully in Latin is 'id est'. "I.e." is used in place of "in other words," or "it/that is." It specifies or makes more clear.
"E.g." means "for example" and comes from the Latin expression exempli gratia, "for the sake of an example," with the noun exemplum in the genitive (possessive case) to go with gratia in the ablative (prepositional case). "E.g." is used in expressions similar to "including," when you are not intending to list everything that is being discussed.
I.e. and e.g. are such common Latin abbreviations that they do not require italicization.
If the form "I.e." looks odd, it's because both "i.e." and "e.g" are usually mid-sentence, surrounded by commas, so they are unlikely to be seen with sentence initial capitals.
Someone quoted this article to insist that i.e. should not be capitalized. The writer began his sentence with "i.e." Evidently I wasn't clear enough that rules about sentence capitalization (or other things) may take precedence over the fact that i.e. isn't generally capitalized. I was only trying to explain why "I.e" might look funny to readers.