Multimedia dedicated weblog.

How to make a DirectShow Muxer Filter - Part 2

October 4th, 2008 Posted in DirectShow

Welcome to the second part of the DirectShow muxer filter tutorial. In this article we will implement a real muxer filter that will be capable of writing a flash video file (FLV).

Prerequisites

I assume you have read the How to make a DirectShow Muxer Filter - Part 1 article and you understand the mechanism how our baseclass works. If you haven’t done so please read the Part 1 tutorial first. You might also consider reading the FLV file format specification which can be found at the Adobe Developer Resources website.

The concept

Our task can be divided into several smaller steps. First we need to take care of supported media types so the muxer can be a part of a reasonable graph. Once the muxer has been fed with some encoded samples we need to format the FLV packets properly. And finally write them out to the output file. For each of these tasks there will be one class to perform it.

Fig. 1. Class diagram

Input streams

The FLV file format is a very simple one and allows only few types of codecs and only one video/audio stream per file. To meet this restriction we must make sure that there will never be more than 2 streams connected at the input of our muxer and that there can be only one of each type (audio or video).

class CFLVMux : public CBaseMux, …
{
public:

    …
    bool                has_video;
    bool                has_audio;
    …

public:
    …

    // custom pins
    virtual int CreatePin(CBaseMuxInputPin **pin, LPCWSTR name);
    virtual HRESULT SetInputType(CBaseMuxInputPin *pin, const CMediaType *pmt);
    virtual HRESULT BreakInputConnect(CBaseMuxInputPin *pin);

    // handling media types
    virtual HRESULT CheckInputType(const CMediaType *pmt);

    …
};

Fig. 2. Input stream handling

Overriding the CBaseMux::CreatePin method will stop the muxer from adding more than 2 pins.

int CFLVMux::CreatePin(CBaseMuxInputPin **pin, LPCWSTR name)
{
    if (!pin) return -1;

    if (pins.size() >= 2) {
        // we don’t want to have any more pins… 2 is perfectly enough for us
        return -1;
    }

    HRESULT                  hr = NOERROR;
    CBaseMuxInputPin    *new_pin = new CBaseMuxInputPin(_T("InPin"), this, &hr, name);

    if (!new_pin) {    *pin = NULL; return -1;    }
    *pin = new_pin;
    return 0;
}

Fig. 3. Limiting the number of pins

A simple modification to the CBaseMux::CheckInputType method will reject connections if multiple streams of the same type were to be connected.

HRESULT CFLVMux::CheckInputType(const CMediaType *pmt)
{
    /* We only support a few types… */
    if (pmt->majortype == MEDIATYPE_Video) {

        // reject if there already is one
        if (has_video) return -1;

        if (pmt->subtype == MEDIASUBTYPE_FLV1) return 0;
    } else
    if (pmt->majortype == MEDIATYPE_Audio) {

        // reject if there already is one
        if (has_audio) return -1;

        if (pmt->subtype == MEDIASUBTYPE_MP3) return 0;
    }
    return -1;
}

Fig. 4. Rejecting streams of the same type

Two more methods need to be modified to make this work - CBaseMux::SetInputType and CBaseMux::BreakInputConnect.

HRESULT CFLVMux::SetInputType(CBaseMuxInputPin *pin, const CMediaType *pmt)
{
    HRESULT        hr = __super::SetInputType(pin, pmt);
    if (FAILED(hr)) return hr;

    // we only want one of each kind
    if (pmt->majortype == MEDIATYPE_Video) {
        has_video = true;

        …

    } else
    if (pmt->majortype == MEDIATYPE_Audio) {
        has_audio = true;

        …
    }

    return NOERROR;
}

HRESULT CFLVMux::BreakInputConnect(CBaseMuxInputPin *pin)
{
    HRESULT        hr = __super::BreakInputConnect(pin);
    if (FAILED(hr)) return hr;

    has_video = false;
    has_audio = false;

    // scan through all our streams and update the has_*** values
    for (int i=0; i<pins.size(); i++) {
        CBaseMuxInputPin    *inpin = pins[i];
        if ((pin != inpin) && inpin->IsConnected()) {
            CMediaType    &mt = inpin->CurrentMediaType();

            if (mt.majortype == MEDIATYPE_Video)    has_video = true;
            if (mt.majortype == MEDIATYPE_Audio)    has_audio = true;
        }
    }

    return NOERROR;
}

Fig. 5. Rejecting streams of the same type

The output class

Each FLV file contains a metadata packet at the beginning of the file which contains information such as duration, filesize or video dimensions. Most of these values can be written at start but several (e.g. duration or filesize) need to be updated after the whole file was written. That’s why we need to be able to get the current position (or size) of the output file and to seek.

The FLVIO class will look like this :

class FLVIO
{
public:

    CComPtr<IStream>    stream;
    __int64                       position;

public:
    FLVIO();
    virtual ~FLVIO();

    void Attach(IStream *str);
    void Detach();

    __int64 Write(BYTE *buffer, int size);
    __int64 Seek(__int64 offset);
    __int64 GetPosition();

};

Fig. 6. FLVIO class

The FLV formatting class

This is the place where the bytestream for the output file will be created. I will not go into much deail considering how the FLV packets look like or what their syntax/semantics is. If you are interested you should read some of the specifications available at Adobe website. For the purpose of this tutorial it is the interface of the FLVWriter class that matters.

class FLVWriter
{
public:

    …

    // I/O object
    FLVIO        *io;

public:
    FLVWriter();
    virtual ~FLVWriter();

    // stream config
    int Reset();
    int ConfigStream(CMediaType *pmt);
    int Start(FLVIO *flvio);
    int Stop();

    // write the data
    int WritePacket(CMuxInterPacket *packet, CMediaType *pmt);

    // metadata helpers
    int WriteMetaData();
};

Fig. 7. FLVWriter class

The methods of this class should be used in the following order:

  1. At muxing start
    • FLVWriter::Reset to reset all internal states
    • FLVWriter::ConfigStream for each valid stream
    • FLVWriter::Start to write the FLV file header
  2. For each muxed packet
    • FLVWriter::WritePacket to write the content of the given packet
  3. At muxing stop
    • FLVWriter::Stop to write the FLV file trailer and to update the metadata packet
    • FLVWriter::Reset to make sure everything has been cleared

int CFLVMux::OnMuxPacket(CMuxInterPacket *packet, CBaseMuxInputPin *inpin)
{
    CAutoLock    lck(&lock_mux);

    if (flv.io) {
        CMediaType    &mt = inpin->CurrentMediaType();
        flv.WritePacket(packet, &mt);
    }
    return 0;
}

int CFLVMux::OnStartStreaming()
{
    /*
        Restart the FLV Muxer instance
    */
    CAutoLock    lck(&lock_mux);

    if (!outstream) return -1;

    flv.Reset();
    flvio.Detach();

    // configure each stream
    for (int i=0; i<2; i++) {
        if (pins[i]->IsConnected()) {
            CMediaType    &mt = pins[i]->CurrentMediaType();
            flv.ConfigStream(&mt);
        }
    }

    // and write something …
    flvio.Attach(outstream);
    flv.Start(&flvio);

    return 0;
}

int CFLVMux::OnStopStreaming()
{
    CAutoLock    lck(&lock_mux);

    /*
        If there is an active muxer, finish the file.
    */

    if (flv.io) {
        flv.Stop();
        flvio.Detach();
        flv.Reset();
    }

    return 0;
}

Fig. 8. Writing the FLV file

Testing

To test the functionality of the muxer filter you will need something capable of producing MP3 and FLV1 (Sorenson Spark) streams. Usually a "ffdshow video encoder" and "LAME Audio Encoder" filters should do the job fine. You can find them both e.g. in the K-Lite Codec Pack.

You might build a graph like the one on the picture.

Fig. 9. Encoding graph

Make sure you have set the video encoder correctly to produce FLV1 stream.

… and the LAME encoder as well.

After you’re done with your settings you might display the muxer property page and enjoy the show :)

 

Conclusion

I hope I have made some things a bit more clear. There are lot of things in this tutorial that have not been covered as deeply (e.g. the filter property page or FLV syntax itself) but if you have made it this far I believe you should be able to find all those details yourself.

The full source code to this filter is also available in the local SVN repository.

Binary download : mmflvmux.ax (350 KB)

Sources checkout : svn co svn://dev.monogram.sk/public/flv_mux/trunk

Enjoy,

Igor

  1. 12 Responses to “How to make a DirectShow Muxer Filter - Part 2”

  2. By tulup on Dec 29, 2008

    hello,
    svc co svn://dev.monogram.sk/public/flv_mux/trunk
    the result was - about 29 Mb of whole ffmpeg project. thanks :\

  3. By Igor Janos on Dec 29, 2008

    Don’t know what you’re talking about.

    janos@media:~/software/flvmux$ svn co svn://dev.monogram.sk/public/flv_mux/trunk .
    A mux_flv.aps
    A mux_flv.vcproj
    A mux_flv.rc
    A src
    A src/stdafx.h
    A src/flv_types.h
    A src/as_objects.h
    A src/flv_filter.h
    A src/flv_writer.cpp
    A src/basemux.cpp
    A src/flv_reg.cpp
    A src/flv_prop.cpp
    A src/flv_writer.h
    A src/basemux.h
    A src/as_objects.cpp
    A src/flv_filter.cpp
    A src/flv_prop.h
    A bin
    A bin/mmflvmux.ax
    A bitmap_logo.bmp
    A mux_flv.sln
    A mux_flv.def
    A resource.h
    Checked out revision 2.
    janos@media:~/software/flvmux$

  4. By Tom on Feb 4, 2009

    Hi Igor,

    Very nice tutorial, thanks. Have you done anything with RTMP or RTMPS, or have any examples?

    Cheers,

    Tom.

  5. By Igor Janos on Feb 4, 2009

    Yes. We’ve done some things with RTMP protocol. What do you have on mind ?

  6. By Sathish on Feb 5, 2009

    I am very new to this area, sorry for this lame question.

    I installed your mmflvmux.ax file and I couldn’t get it working in GraphEdit as per ur diagram.

    Can you tell me where you picked your ffdshow encoder and decoder dlls from, I see that my dlls are different and don’t seem to match this scenario.

  7. By Igor Janos on Feb 5, 2009

    I’ve installed the latest K-Lite codec pack.
    http://www.codecguide.com

  8. By Tom on Apr 30, 2009

    Hi again Igor,
    been a bit busy, sorry for the tardiness of my response. I am now getting back around to the RTMP stuff I was talking about. I am hoping to write a filter which broadcasts FLV using RTMP on a specified TCP port. Shouldn’t be too hard do you think? Any RTMP header/encoding examples much appreciated!

  9. By jirong on Jul 29, 2009

    Tom and Igor,

    Any progress on this topic?

    “I am hoping to write a filter which broadcasts FLV using RTMP on a specified TCP port. Shouldn’t be too hard do you think? Any RTMP header/encoding examples much appreciated!”

  10. By polo on Sep 22, 2009

    I got some problem while building the whole project .

    These files are missing on the SVN :

    “basemux.h”
    “basemux.cpp”

    would you please send these files to my e-mail address ?

    Thanks~

  11. By polo on Sep 22, 2009

    sorry , I found it on the other svn

  12. By Sumit on Oct 12, 2009

    Hi Igor,

    When i connect 2 different file source, it interleaves 10-15 times. After that the data from one pin keep on coming. I have checked the file size of second file also,both are of equal size.

  13. By SoniX on Oct 19, 2009

    Here is the link to RTMP specifications:

    http://www.adobe.com/devnet/rtmp/

    Regards…

Post a Comment