Notes on writing a VMR-9 custom allocator/presenter in C# PDF Print E-mail
The latest release of the Video Mixing Renderer (VMR-9) offers very interesting possibilities. It has a modular design that allows developpers to replace its main components by custom made components. We can replace the Allocator/Presenter and the Compositor component. In these notes, I'll describe how to write a custom allocator/presenter in C#. The code is not as well polished and robust as I would like it. But it should serve as a good starting point for others to build upon. Once again, the code can be found in a zip file.

The documentation for this task is contained in a single page of the DirectShow SDK documentation entitled "Supplying a Custom Allocator-Presenter for VMR-9". The VMR9Allocator sample (in the VMR-9 samples directory) was used as the base for our implementation.

First, we have to deal with accessing the required interfaces in C#. The DirectShowLib contains these definition in their "untested" sections. I looked at these, and I decided to create my own custom library because the "untested" version of IVMRSurfaceAllocatorNotify9.AllocateSurfaceHelper had an object for last argument, and I really prefer just a plain old IntPtr. So I wrote a small idl file with the needed interfaces which contain the following (plus a few that we won't need, all this taken from customap.idl)

  

  1.     
  2. interface IGraphBuilder;
  3. interface IMediaControl;
  4. interface IMediaEventEx;
  5. interface IMediaSeeking;
  6. interface IVMRFilterConfig9;
  7. interface IVMRMixerControl9;
  8. interface IFileSinkFilter;
  9. interface IVMRSurfaceAllocator9;
  10. interface IVMRImagePresenter9;
  11. interface IVMRSurfaceAllocatorNotify9;
  12. interface IVMRMonitorConfig9;
  13.  
  

During the development, I had to modify the standard marshaler behaviour for IVMRSurfaceAllocatorNotify9.SetD3DDevice because it creates this method with ref uint arguments and the only way I found to provide an interface pointer to the device needed for the first argument of this method was through the property Device.UnmanagedComPointer (which require the /unsafe switch to the C# compiler). The modification is minor and was created as follows: I ran the build_customaptypelib command file to produce an interop assembly for the interfaces listed above. I disassembled the assembly just produced (with the command "ildasm /out="tmp.il" customaptypelib.dll" and replace, in the file "tmp.il", the two arguments to SetD3DDevice with native int. Then I reassembled the assembly with the command: ilasm /dll /output=customaptypelib.dll" tmp.il. These steps are well explained in the section entitled "Editing An Interop Assembly" of the .Net SDK documentation but it makes you wonder why Microsoft doesn't provide a type library (.tlb) file for DirectShow. Now, we are ready to actually implement our custom allocator/presenter.

  

  1.  
  2. using Microsoft.DirectX;
  3. using Microsoft.DirectX.Direct3D;
  4. using CustomAPTypeLib;
  5.  
  

The VMR-9 renderer will hand over video samples as Direct3D surfaces to our custom allocator/presenter, so we need access to Direct3D and we reference these assemblies (plus our own CustomAPTypeLib). Then, we declare variables for the interfaces that we haven't seen before: 

  1.     
  2. IVMRFilterConfig9 fc = null;
  3. IVMRSurfaceAllocatorNotify9 san = null;
  4. CustomAP customAP = null;
  5.  
  

The interface IVMRFilterConfig9 is needed to set the VMR-9 filter in "renderless" mode i.e. the filter will not render samples but will let our "presenter" component do the job. The IVMRSurfaceAllocatorNotify9 interface is implemented on the VMR-9 filter and allows the renderer and our component to communicate. The customAP variable is our custom allocator/presenter component that implement two interfaces; it is declared as follows: (taken from CustomAP.cs) 

  1.   
  2. public class CustomAP : IVMRSurfaceAllocator9, IVMRImagePresenter9
  3.  
  

We'll come back to these two interfaces a little later. In the function VMR9Alloc which build the filter graph, we find:

  

  1.  
  2. // create the vmr object 
  3. t = Type.GetTypeFromCLSID( Vmr9Guid );
  4. vmr = Activator.CreateInstance( t );                
  5. // configure the vmr in "renderless" mode
  6. fc = (IVMRFilterConfig9)vmr;          
  7. fc.SetRenderingMode( 0x00000004 ); //VMRMode_Renderless  = 0x00000004
  8. // create our custom a/p
  9. customAP = new CustomAP( this );
  10. // tell the renderer about our allocator               
  11. san = (IVMRSurfaceAllocatorNotify9)vmr;
  12. san.AdviseSurfaceAllocator( 0, customAP );
  13. // tell our AP about the render
  14. IVMRSurfaceAllocator9 sa = (IVMRSurfaceAllocator9)customAP;
  15. sa.AdviseNotify( san );
  16. gb.AddFilter( (IBaseFilter)vmr, "Video Mixing Renderer9" );
  17.  
  

We create the VMR-9 renderer and access its IVMRFilterConfig9 interface to call the method SetRenderingMode with a "renderless" argument. Then we create our custom A/P and tell the renderer about it using the interface IVMRSurfaceAllocatorNotify9 and the method AdviseSurfaceAllocator. While it's not clear from the documentation, accessing this interface on the renderer should be done after setting the renderer in "renderless" mode because some cards will not allow it otherwise (I learned it the hard way :-| We tell our component about the renderer with a call to AdviseNotify and let "Intelligent Connect" do its magic.

We are now ready to look at the implementation of our allocator/presenter component. Its member variables are as follows: 

  1.     
  2. // our direct3d device
  3. Device device = null;
  4. // we need to remember our renderer
  5. IVMRSurfaceAllocatorNotify9 m_san = null;
  6. // a ref to our main window (needed in when we create a device)
  7. Form m_window = null;
  8. // our render target surface
  9. Surface m_renderTarget = null;
  10. // an array of pointer to surfaces we allocate
  11. IntPtr [] surfaces = null;
  12. // our object to draw a scene
  13. Scene m_scene = null;
  14. // a guid for the Direct3DTexture interface
  15. Guid IID_IDirect3DTexture9;
  16.  
  

The comments make these declarations fairly straightforward. Then we have the method CreateDevice

  

  1.     
  2. void CreateDevice() 
  3. {
  4. AdapterInformation ai = Manager.Adapters[0];
  5. DisplayMode dm = ai.CurrentDisplayMode;
  6. PresentParameters pp = new PresentParameters();
  7. pp.Windowed = true;
  8. pp.DeviceWindow = m_window;
  9. pp.SwapEffect = SwapEffect.Copy;
  10. pp.BackBufferFormat = dm.Format;
  11. device = new Device( 0, DeviceType.Hardware, m_window, 
  12. CreateFlags.SoftwareVertexProcessing, pp );
  13. device.DeviceResizing += new CancelEventHandler( cancelResizing );
  14. m_renderTarget = device.GetRenderTarget( 0 );
  15. }
  16.  
  

This is a direct translation of the C++ code in the VMR9Allocator sample. The only difference is the new handler for cancelling the DeviceResizing event. The vertex buffer for our rectangle, in the Scene class, will be declared with the Pool.Managed as argument and we'll be happy with the default DirectX behaviour (instead of the smarter processing done by Managed DirectX).

The DirectShow documentation says on point 6 (for supplying a custom A/P): "Allocate surfaces in the IVMRSurfaceAllocator9::InitializeDevice callback method". So this what we are going to do next. We start with:

  

  1.         
  2. uint width = 1, height = 1;
  3. while( width < lpAllocInfo.dwWidth ) width = width << 1;
  4. while( height < lpAllocInfo.dwHeight ) height = height << 1;
  5. float fTU = 1.0f, fTV = 1.0f;
  6. fTU = (float)(lpAllocInfo.dwWidth) / (float)(width);
  7. fTV = (float)(lpAllocInfo.dwHeight) / (float)(height);
  8. m_scene.SetSrcRect( fTU, fTV );
  9. lpAllocInfo.dwWidth = width;
  10. lpAllocInfo.dwHeight = height;
  11.  
  

Because textures whose dimensions are not power of 2 suffer significant performance penalties on many cards, we fix the dimension to satisfy this requirement. We also arrange our texture coordinates to represent our new dimension. Following this, we make sure that we are requesting "texture" surfaces and call AllocateSurfaceHelper on the IVMRSurfaceAllocatorNotify9 of the renderer. 

  1.         
  2. // VMR9AllocFlag_TextureSurface  = 0x0004
  3. lpAllocInfo.dwFlags |= 0x0004;
  4. DeleteSurfaces();
  5. // allocate memory for our surface pointers
  6. IntPtr buffer = Marshal.AllocCoTaskMem( Marshal.SizeOf( typeof( IntPtr )) * 
  7. (int)lpNumBuffers );        
  8. // ask the renderer to create these surfaces
  9. m_san.AllocateSurfaceHelper( ref lpAllocInfo, ref lpNumBuffers, buffer );
  10. // retrieve the pointers and stuff them in our array
  11. surfaces = new IntPtr[lpNumBuffers];
  12. for( int i = 0; i < lpNumBuffers; i++ ) 
  13. {
  14. // following only work on 32 bit machines
  15. surfaces[i] = Marshal.ReadIntPtr(buffer , i*4);
  16. }
  17. Marshal.FreeCoTaskMem( buffer );
  18.  
  

We are keeping pointers to surfaces in our surfaces array which is sufficient for our purpose; so we just read the values returned by AllocateSurfaceHelper and stuff them in our array. After that, we call Init on our Scene

The implementation of GetSurface just returned one of the pointers returned previously: 

  1.     
  2. // IVMRSurfaceAllocator9.GetSurface
  3. public void GetSurface ( UInt32 dwUserID , 
  4. UInt32 SurfaceIndex , 
  5. UInt32 SurfaceFlags , 
  6. IntPtr lplpSurface )
  7. {
  8. if( SurfaceIndex > surfaces.Length )
  9. throw new Exception( "can't get surface" );
  10. lock( this ) 
  11. {
  12. Marshal.WriteIntPtr( lplpSurface, surfaces[SurfaceIndex] );
  13. }
  14. }
  15.  
  

Since lplpSurface points in native memory, we call WriteIntPtr to copy our surface pointer into this location. AdviseNotify stores away the pointer to our renderer and asks it to SetD3DDevice: 

  1.     
  2. // IVMRSurfaceAllocator9.AdviseNotify
  3. public unsafe void AdviseNotify ( IVMRSurfaceAllocatorNotify9 lpIVMRSurfAllocNotify )
  4. {      
  5. m_san = lpIVMRSurfAllocNotify;
  6. AdapterInformation ai = Manager.Adapters.Default;
  7. IntPtr hMonitor = Manager.GetAdapterMonitor(ai.Adapter);
  8. lock( this ) 
  9. {
  10. m_san.SetD3DDevice( (IntPtr)device.UnmanagedComPointer, hMonitor );            
  11. }
  12. }
  13.  
  

I haven't found a better way to do this in C#. PresentImage is called by the renderer when it wants our component to display a video frame (or sample). The code simply sets the rendering target, gets the texture from the surface field of the PresentationInfo structure, calls our Scene.DrawScene and finally dump it to the screen with Device.Present. 

  1.     
  2. Surface s = null;
  3. Texture texture = null;
  4. // IVMRImagePresenter9.PresentImage
  5. public void PresentImage ( System.UInt32 dwUserID , ref _VMR9PresentationInfo lpPresInfo )
  6. {
  7. // should check args and lock
  8. if( lpPresInfo.lpSurf == IntPtr.Zero )
  9. throw new Exception( "Presentation Info corrupted" );
  10. lock( this ) 
  11. {
  12. device.SetRenderTarget( 0, m_renderTarget );
  13. s = new Surface( lpPresInfo.lpSurf );              
  14. texture = (Texture)s.GetContainer( IID_IDirect3DTexture9 );      
  15. m_scene.DrawScene( device, texture );
  16. device.Present();
  17. texture.Dispose();
  18. s.Dispose();
  19. }
  20. }
  21.  
  

The Scene class is standard Managed DirectX and we'll skip the details.

[Note (25/9/06), I have received the following message. When I'll update this document, I'll look at. Till then, it might be of some help.

Hello,

I'm sending you some informations related to playing .MPG and .WMV files using your useful C# implementation of custom VMR-9 allocator/presenter (tutorialDS7.zip). Playing .AVI files worked from start but when I tried to play .MPG or .WMV files, there were some problems.

After some poking around I found out a way to play these type of files as well. For example, addition to VidTexture.cs:

 

... fc = (IVMRFilterConfig9)vmr; fc.SetNumberOfStreams(1); fc.SetRenderingMode( 0x00000004 ); //VMRMode_Renderless = 0x00000004 ...

Consequence of this addition is that lpAllocInfo.Format as argument to IVMRSurfaceAllocator9.InitializeDevice() is always 0 and call to m_san.AllocateSurfaceHelper doesn't fail as in situation when fc.SetNumberOfStreams is not called.

If you think these informations are useful to other users of tutorialDS7.zip, you can post it on your notes page (http://www.a2ii..com/tech/directx/tutorialDS7.htm).

Thanks, Predrag ]

 



Bookmark it...
Digg!Reddit!Del.icio.us!Google!Facebook!Slashdot!Technorati!StumbleUpon!Newsvine!Furl!Yahoo!Ma.gnolia!
 
< Prev   Next >
[ Back ]
Joomla Template by Joomlashack
Joomla Templates by JoomlaShack Joomla Templates by Compass Design