熱點推薦:
您现在的位置: 電腦知識網 >> 編程 >> .NET編程 >> 正文

Out Of Memory的分析及診斷方法

2013-11-13 10:07:24  來源: .NET編程 

  首先什麼是Out Of Memory?就是內存溢出簡稱OOM(下邊我就用這個簡稱了啊!)說白了就是程序想用內存的時候OS沒有那麼多內存可以分配了然後就抱OOM錯誤了

  首先介紹一下我這個項目的情況基於exchange +sp+hmc+web service call通過一個winform的模擬測試程序單線程添加信息循環萬次每循環一次創建一個公司開通郵件域名並創建個帳號每個帳號都開通郵件服務現在循環到次左右的時候wwpexe的內存占用為private bytesMvirtual bytesM聽兄弟講他們做過類似的測試當循環到個的時候會出現OOM的問題

  既然是OOM我們當然要介紹一個超級cool的工具debugdiag!(這個工具以後再介紹因為要貼N多圖實在痛苦……)通過debugdiag抓memory leak的dump(M)發現有如下問題mscorwks洩漏了M左右的內存

  現在轉到windbg中來我們首先看命令!eeheap它一共有兩個參數

  > !help eeheap

  

  !EEHeap [gc] [loader]

  首先看一下gc的參數!eeheap gc這個命令表明我們程序占用托管堆的大小

  > !eeheap gc

  Number of GC Heaps

  generation starts at xfff

  generation starts at xfdfc

  generation starts at xc

  ephemeral segment allocation contextxbcbf xbdc)

  segment begin allocated size

  ead acc ad xedc(

  ea da fb xfb

  c c bdc xfcc(

  Large object heap starts at xc

  segment begin allocated size

  c c cfbbd xabd

  Total Size xad

  

  GC Heap Size xad

  dump大小為M托管堆大小為M不到差別很大的!剩下的內存在哪裡?現在來看一個新的命令!address運行後會有一坨又一坨的輸出我們關心的是後面的summary如下

   Usage SUMMARY

  TotSize ( KB) Pct(Tots) Pct(Busy) Usage

  b % % RegionUsageIsVAD

  cb % % RegionUsageFree

  d % % RegionUsageImage

  fc % % RegionUsageStack

   % % RegionUsageTeb

   % % RegionUsageHeap

   % % RegionUsagePageHeap

   % % RegionUsagePeb

   % % RegionUsageProcessParametrs

   % % RegionUsageEnvironmentBlock

  Tot fff KB) Busy bc KB)

   Type SUMMARY

  TotSize ( KB) Pct(Tots) Usage

  cb % <free>

  c % MEM_IMAGE

  a % MEM_MAPPED

  ad % MEM_PRIVATE

   State SUMMARY

  TotSize ( KB) Pct(Tots) Usage

  ee % MEM_COMMIT

  cb % MEM_FREE

  ca % MEM_RESERVE

  上面的信息比較有意思RegionUsageImage代表的是dlls占用的內存一共是MRegionUsageheap代表的是NT heaps一共是MMEM_COMMIT和MEM_RESERVE加起來是virtual memory他倆的合計是M我們還少看了什麼?!eeheap還有一個參數loader運行一下後會有N長的結果我們看一部分

  !eeheap –loader

  Domain fbd

  LowFrequencyHeap badeebeaff) 很黃很暴力…… edcee) Size xce)bytes

  Wasted x)bytes

  HighFrequencyHeap baefaacfd) 很黃很暴力 ebaee) Size xa)bytes

  Wasted x)bytes

  StubHeap baa) Size x)bytes

  Virtual Call Stub Heap

  IndcellHeap Size x)bytes

  LookupHeap Size x)bytes

  ResolveHeap Size x)bytes

  DispatchHeap Size x)bytes

  CacheEntryHeap bc) Size x)bytes

  Total size xc)bytes

  一共占用了M內存繼續看下面的內容我居然發現了個module!!!

  Module Thunk heaps

  Module aa Size x)bytes

  Module e Size x)bytes

  Module a Size x)bytes

  Module Size x)bytes

  =============很黃很暴力======================

  Module ee Size x)bytes

  Module ee Size x)bytes

  Module eec Size x)bytes

  Module eec Size x)bytes

  Total size x)bytes

  

  Total LoaderHeap size xaf)bytes

  =======================================

  問題基本出來了居然在內存裡面有將近萬個module!隨便dump出來一個看看這裡又有一個命令!dumpmodule

  隨便dump出來一個看類似如下信息

  > !dumpmodule ee

  Name qrtxcw Version= Culture=neutral PublicKeyToken=null

  Attributes PEFile

  Assembly cfe

  LoaderHeap

  TypeDefToMethodTableMap edcc

  TypeRefToMethodTableMap edccc

  MethodDefToDescMap edccc

  FieldDefToDescMap edcc

  MemberRefToDescMap edccc

  FileReferencesMap edcc

  AssemblyReferencesMap edcc

  MetaData start address ed bytes)

  這裡可能看不到啥那麼我們加一個參數mt來看看

  > !dumpmodule mt ee

  Name qrtxcw Version= Culture=neutral PublicKeyToken=null

  Attributes PEFile

  Assembly cfe

  LoaderHeap

  TypeDefToMethodTableMap edcc

  TypeRefToMethodTableMap edccc

  MethodDefToDescMap edccc

  FieldDefToDescMap edcc

  MemberRefToDescMap edccc

  FileReferencesMap edcc

  AssemblyReferencesMap edcc

  MetaData start address ed bytes)

  Types defined in this module

  MT TypeDef Name

  

  eec x MicrosoftXmlSerializationGeneratedAssemblyXmlSerializationReaderCreateUserResponseData

  eec x MicrosoftXmlSerializationGeneratedAssemblyXmlSerializerContract

  Types referenced in this module

  MT TypeRef Name

  

  eeac x SystemXmlSerializationXmlSerializationReader

  eb x SystemXmlSerializationXmlSerializerImplementation

  ebc x MicrosoftProvisioningWebServicesHostedActiveDirectoryCreateUserResponseData

  ebe x SystemXmlXmlReader

  fdcc x SystemCollectionsHashtable

  fa xe SystemObject

  ec x SystemXmlXmlQualifiedName

  c x SystemBoolean

  eab x SystemXmlXmlNameTable

  出現了SystemXmlSerialization大家熟悉嗎?我們轉過頭來看debugdiag分析的call stack

  Call stack sample

  Address xc

  Allocation Time since tracking started

  Allocation Size Bytes

  Function Source Destination

  mscorjit!norls_allocatornraAllocNewPage+

  mscorjit!norls_allocatornraAlloc+ mscorjit!norls_allocatornraAllocNewPage

  mscorjit!jitNativeCode+ mscorjit!norls_allocatornraAlloc

  mscorjit!CILJitcompileMethod+d mscorjit!jitNativeCode

  xECE

  mscorjit!CompilerimpExpandInline+aa

  mscorjit!CompilerfgMorphTree+

  mscorjit!CompilerfgMorphStmts+ mscorjit!CompilerfgMorphTree

  mscorjit!CompilerfgMorphBlocks+ mscorjit!CompilerfgMorphStmts

  mscorjit!CompilerfgMorph+ mscorjit!CompilerfgMorphBlocks

  mscorjit!CompilercompCompile+f mscorjit!CompilerfgMorph

  mscorjit!CompilercompCompile+d mscorjit!CompilercompCompile

  mscorjit!jitNativeCode+b mscorjit!CompilercompCompile

  mscorjit!CILJitcompileMethod+d mscorjit!jitNativeCode

  xEEDFF

  SystemXmlSerializationTempAssemblyInvokeReader(SystemXmlSerializationXmlMapping SystemXmlXmlReader SystemXmlSerializationXmlDeserializationEvents SystemString)

  SystemXmlSerializationTempAssemblyInvokeReader(SystemXmlSerializationXmlMapping SystemXmlXmlReader SystemXmlSerializationXmlDeserializationEvents SystemString)

  SystemXmlSerializationXmlSerializerDeserialize(SystemXmlXmlReader SystemString SystemXmlSerializationXmlDeserializationEvents) SystemXmlSerializationTempAssemblyInvokeReader(SystemXmlSerializationXmlMapping SystemXmlXmlReader SystemXmlSerializationXmlDeserializationEvents SystemString)

  SystemXmlSerializationXmlSerializerDeserialize(SystemXmlXmlReader SystemString) SystemXmlSerializationXmlSerializerDeserialize(SystemXmlXmlReader SystemString SystemXmlSerializationXmlDeserializationEvents)

  SystemXmlSerializationXmlSerializerDeserialize(SystemIOStream) SystemXmlSerializationXmlSerializerDeserialize(SystemXmlXmlReader SystemString)

  MicrosoftProvisioningSdkXmlSerializationProvisioningObjectFactoryConvert[[System__Canon mscorlib][System__Canon mscorlib]](System__Canon) SystemXmlSerializationXmlSerializerDeserialize(SystemIOStream)

  MicrosoftProvisioningWebServicesServiceBaseSubmit[[System__Canon mscorlib][System__Canon mscorlib]](System__Canon SystemString SystemString Boolean) MicrosoftProvisioningSdkXmlSerializationProvisioningObjectFactoryConvert[[System__Canon mscorlib][System__Canon mscorlib]](System__Canon)

  MicrosoftProvisioningWebServicesHostedActiveDirectoryServiceCreateOrganization(MicrosoftProvisioningWebServicesHostedActiveDirectoryCreateOrganizationRequest Boolean)

  xEBEB

  xAEE

  SystemWebServicesProtocolsLogicalMethodInfoInvoke(SystemObject SystemObject[])

  SystemWebServicesProtocolsWebServiceHandlerInvoke() SystemWebServicesProtocolsLogicalMethodInfoInvoke(SystemObject SystemObject[])

  xFFC

  SystemThreading_TimerCallbackTimerCallback_Context(SystemObject)

  SystemThreadingExecutionContextRun(SystemThreadingExecutionContext SystemThreadingContextCallback SystemObject)

  webengine!HashtableIUnknownAddCallback+a

  webengine!HttpCompletionProcessRequestInManagedCode+a

  webengine!HttpCompletionProcessRequestInManagedCode+a

  webengine!HttpCompletionProcessCompletion+e webengine!HttpCompletionProcessRequestInManagedCode

  webengine!CorThreadPoolWorkitemCallback+

  xF

  xFBC

  kernel!BaseThreadStart+

  看到這裡基本差不多偶認為是exchange內部的代碼問題此話怎講?從頭說在SystemXmlSerialization下面有一個by design的bug我們用reflector看XmlSerializer的構造代碼

   thistempAssembly = cache[defaultNamespace type]

   if (thistempAssembly == null)

   {

   lock (cache)

   {

   thistempAssembly = cache[defaultNamespace type]

   if (thistempAssembly == null)

   {

   XmlSerializerImplementation implementation

   Assembly assembly = TempAssemblyLoadGeneratedAssembly(type defaultNamespace out implementation)

   if (assembly == null)

   {

   thismapping = new XmlReflectionImporter(defaultNamespace)ImportTypeMapping(type null defaultNamespace)

   thistempAssembly = GenerateTempAssembly(thismapping type defaultNamespace)

   }

   else

   {

   thismapping = XmlReflectionImporterGetTopLevelMapping(type defaultNamespace)

   thistempAssembly = new TempAssembly(new XmlMapping[] { thismapping } assembly implementation)

   }

   }

   cacheAdd(defaultNamespace type thistempAssembly)

   }

   為了加快運行速度xmlserializer做了cache在代碼中也許我們有N多的type那麼每個type在這裡都做了一個assembly都把assembly放到了cache中這樣本來沒問題但是如果type有幾千個有幾萬個那麼就有幾千幾萬個temp assembly出現這些assembly都很小也許只有個字節也許是字節但是注意的是內存分配時是按照來分配的假如說每個塊最小為k大小那麼即使只分配一個字節的內存我們也要申請一個k的page那麼如果我們的小塊非常非常多那麼我們身子縮小N倍後你會發現內存裡面四處都是小窟窿這些窟窿加起來很大但是別人就是不能用因為這k內存必須要連續的塊

  so當我們發現wwpexe僅僅百兆的時候就報OOM了就是這個原因

  關於這個bug可以看msdn這個kbus


From:http://tw.wingwit.com/Article/program/net/201311/12615.html
    推薦文章
    Copyright © 2005-2013 電腦知識網 Computer Knowledge   All rights reserved.