LLMedia API

NOTE: This system is deprecated. Use the new Media Rendering Plugin Framework instead.

LLMedia Documentation

Abstract

The LLMedia API provides a consistent approach to embedding various Rich Media Rendering Engines (Implementations) in a manner consistent with the needs of Second Life. The API provides a framework for linking a URL type (loosely composed of MIME Type and/or Scheme) to the interface layer of the associated Media Engine, and for passing back the rendered OpenGL texture. It also supports a variety of input controls to enable common media navigation paradigms.

Overview

Located in the Second Life client source tree at $BRANCH_NAME/indra/llmedia
Allows consumers of the library to render different types of media to a well defined memory buffer
Specifically, in the Second Life client, this memory buffer is used as the source for OpenGL textures
It is deliberately loosely coupled to the rest of the viewer - no dependencies whatsoever on viewer code
Each media type is defined using its MIME type (if present) or URL scheme (if present)
Different media types are handled by implementation files specific to each type - hereafter called Impls
A media manager class handles registration, creation, management and destruction of Impls
Impls for new media types are easily integrated - write a C++ impl class and tell the media manager class which MIME type or URL scheme you support

Currently supported media types (Impls)

LLMediaImplQuickTime
- Apple QuickTime media
- Broadly speaking, supports everything the Apple QuickTime player does
- Available on Win32 and Mac

LLMediaImplGStreamer
- Supports a wide variety of QuickTime/AVI/other media depending on installed system plug-ins
- Available on Linux

LLMediaImplLLMozLib
- LLMozLib media
- Web based media
- Very broadly speaking, supports everything that Firefox 2.0.x supports
- Notable exceptions are popups, plugins, .htaccess based authorization, feedback dialogs

API class layout

Abstract base class (llMediaBase)
Common Impl class (llMediaImplCommon) derives from this abstract base class and contains a set of matching (non-pure) methods along with some utility functions.
Each Impl derives from llMediaImplCommon and overrides as much of it as it needs to do its work.
Each Impl must implement an llMediaImplMaker and override the create() function. This object is also used to register for mime types and schemes.
A factory class (llMediaImplFactory) is responsible for registering an Impl and then making one as and when required.
A manager class (llMediaManager) manages the list of Impls (including creation/deletion/interrogation) and also contains common utility functions.
An event system based on the well known emitter/observer pattern is implemented via the (llMediaEmitter and llMediaObserver)

API functions

Housekeeping

Function:	`bool init();`
Description:	Local initialization, called by the media manager when creating a media source Different from the global `(Impl)::startUp()` & `(Impl)::closeDown()` static methods which are designed to be called once at application startup and closedown.
Return value:	`true` on success, `false` on failure

Function:	`bool reset();`
Description:	Undoes everything init() did called when the media manager destroys a media source
Return value:	`true` on success, `false` on failure

Function:	`bool setMimeType( const std::string mime_type ); std::string getMimeType() const;`
Description:	Accessor/mutator for MIME type
Return value:	`true` on success, `false` on failure

Function:	`std::string getMediaURL() const;`
Description:	Accessor for current URL
Return value:	The current media URL This may be different from the original URL that was passed to `navigateTo` - a HTTP redirect may have occurred for example

Function:	`std::string getVersion();`
Description:	Get a version string specific to the Impl Format of version string is specific to each Impl
Return value:	The version string

Function:	`bool set404RedirectUrl( std::string redirect_url ); bool clr404RedirectUrl();`
Description:	Set/clear URL to visit when a 404 page is reached This (unusual) functionality was required for Web based Search because Amazon S3 404 pages are XML based and not HTML
Return value:	`true` on success, `false` on failure

Function:	`bool setBackgroundColor( unsigned int red, unsigned int green, unsigned int blue ) const;`
Description:	Sets the background color of the media surface Pass in 0 - 255 for each color component
Return value:	`true` on success, `false` on failure

Function:	`bool setCaretColor( unsigned int red, unsigned int green, unsigned int blue ) const;`
Description:	Sets the color of the caret in media impls that have one - Web related ones do for example. Pass in 0 - 255 for each color component
Return value:	`true` on success, `false` on failure

Media Management

Function:	`bool updateMedia();`
Description:	Tells Impl to update itself Must be needs to be called on a regular basis No guarantee in this version that this will return quickly - this is up to the Impl.
Return value:	`true` on success, `false` on failure

Function:	`bool setRequestedMediaSize( int media_width, int media_height );`
Description:	Request that the Impl change the media height and width to given dimensions in pixels May fail if the Impl doesn't support changing size
Return value:	`true` on success, `false` on failure

Function:	`int getMediaWidth() const;`
Description:	Gets current media width May change throughout lifetime of media stream Note: event is emitted that consumers of this library can catch when media size changes
Return value:	The current media width in pixels

Function:	`int getMediaHeight() const;`
Description:	Gets current media height May change throughout lifetime of media stream Note: event is emitted that consumers of this library can catch when media size changes
Return value:	The current media height in pixels

Function:	`bool setMediaDepth( int media_depth );`
Description:	Request that the Impl change the media depth to given dimensions in bytes May fail if the Impl doesn't support changing size TODO: ought this to be called setRequestedMediaDepth() ?
Return value:	`true` on success, `false` on failure

Function:	`int getMediaDepth() const;`
Description:	Gets current media depth May change throughout lifetime of media stream Note: event is emitted that consumers of this library can catch when media size changes
Return value:	The current media depth in bytes

Function:	`int getMediaBufferSize() const;`
Description:	Gets size of media buffer for current media surface Important: this might NOT be the same as media width * height * depth if the Impl is padding stride length for example
Return value:	The size in bytes of the current media buffer

Function:	`unsigned char* getMediaData();`
Description:	Returns pointer to raw media pixels
Return value:	A pointer to the current media data

Function:	`int getMediaDataWidth() const;`
Description:	Get the width of the media data May be different from the size of the media if the Impl is padding stride length for example
Return value:	The width of the media data in pixels

Function:	`int getMediaDataHeight() const;`
Description:	Get the height of the media data May be different from the size of the media if the Impl is padding height This is almost always the same as media height
Return value:	The height of the media data in pixels

Texture Management

Function:	`int getTextureFormatInternal() const;`
Description:	Gets internal format to use for OpenGL texture
Return value:	A value for internal texture format that mirrors the OpenGL version Valid values are: `LL_MEDIA_RGB` `LL_MEDIA_RGBA` `LL_MEDIA_RGB8` `LL_MEDIA_BGR` `LL_MEDIA_BGRA`

Function:	`int getTextureFormatPrimary() const;`
Description:	Gets primary format to use for OpenGL texture
Return value:	A value for primary texture format that mirrors the OpenGL version Valid values are: `LL_MEDIA_RGB` `LL_MEDIA_RGBA` `LL_MEDIA_RGB8` `LL_MEDIA_BGR` `LL_MEDIA_BGRA`

Function:	`int getTextureFormatType() const;`
Description:	Gets format type to use for OpenGL texture
Return value:	A value for texture format type that mirrors the OpenGL version Valid values are: `LL_MEDIA_UNSIGNED_BYTE` `LL_MEDIA_UNSIGNED_INT_8_8_8_8` `LL_MEDIA_UNSIGNED_INT_8_8_8_8_REV`

Audio

Function:	`bool setVolume( float volume ); float getVolume() const;`
Description:	Accessor/mutator for volume from media stream if present Volume level is 0.0 (off) to 1.0 (maximum)
Return value:	`true` on success, `false` on failure

Transport Control

Function:	`bool addCommand( ECommand cmd );`
Description:	Add a transport command to the list of ones to be processed Note: Currently the list is only 1 entry long so adding a command will remove the existing one Supported commands are currently: `ECommand::COMMAND_STOP` - Stop media stream if appropriate `ECommand::COMMAND_START` - Start media stream if appropriate `ECommand::COMMAND_PAUSE` - Pause media stream if appropriate `ECommand::COMMAND_BACK` - Go back to last location in media stream if appropriate `ECommand::COMMAND_FORWARD` - Go forward to next location in media stream if appropriate
Return value:	`true` on success, `false` on failure

Function:	`bool clearCommand();`
Description:	Remove a transport command from the list of ones to be processed Note: Currently the list is only 1 entry long so clearing a command will empty the list
Return value:	`true` on success, `false` on failure

Function:	`bool updateCommand();`
Description:	Tells transport command queue to update itself Must be needs to be called on a regular basis
Return value:	`true` on success, `false` on failure

Function:	`EStatus getStatus();`
Description:	Returns a state code that describes what the current transport status is Supported values are currently: `EStatus::STATUS_UNKNOWN` - Status code is unknown `EStatus::STATUS_INITIALIZING` - media source is initializing `EStatus::STATUS_NAVIGATING` - media source is navigating to a new URL `EStatus::STATUS_STARTED` - media source has been started `EStatus::STATUS_STOPPED` - media source has been stopped `EStatus::STATUS_PAUSED` - media source has been paused `EStatus::STATUS_RESETTING` - media source is resetting (uninitializing)
Return value:	`true` on success, `false` on failure

Function:	`bool seek( double time );`
Description:	Seeks to a location within a media source stream Time is in seconds since the start of the stream Only meaningful for video type streams like QuickTime for example
Return value:	`true` on success, `false` on failure

Function:	`bool setLooping( bool enable);`
Description:	Turn looping on or off. If looping is on, media will restart from the beginning when it reaches the end If looping is off, media still pause on the last frame when it reaches the end
Return value:	`true` on success, `false` on failure

Function:	`bool isLooping();`
Description:	Determine if looping is switched on or off
Return value:	`true` if looping is enabled, `false` if looping is disabled

Scaling

Function:	`bool setAutoScaled( bool auto_scaled );`
Description:	Turn media autoscaling on or off Autoscale means try to scale media to size of texture May fail if media doesn't support size change
Return value:	`true` on success, `false` on failure

Function:	`bool isAutoScaled() const;`
Description:	Determine if autoscaling is turned on or off
Return value:	`true` if autoscaling is turned on, `false` if autoscaling is turned off

Mouse and Keyboard

Function:	`bool mouseDown( int x_pos, int y_pos );`
Description:	Inject left mouse button press at given X/Y coordinate within media surface
Return value:	`true` on success, `false` on failure

Function:	`bool mouseUp( int x_pos, int y_pos );`
Description:	Inject left mouse button release at given X/Y coordinate within media surface
Return value:	`true` on success, `false` on failure

Function:	`bool mouseLeftDoubleClick( int x_pos, int y_pos );`
Description:	Inject left mouse button double click at given X/Y coordinate within media surface
Return value:	`true` on success, `false` on failure

Function:	`bool mouseMove( int x_pos, int y_pos );`
Description:	Inject mouse cursor move to given X/Y coordinate within media surface
Return value:	`true` on success, `false` on failure

Function:	`bool keyPress( int key_code );`
Description:	Inject key press with given ASCII value into media surface No support currently for discrete key down/up events
Return value:	`true` on success, `false` on failure

Function:	`bool unicodeInput( unsigned long uni_char );`
Description:	Inject key press with given UNICODE value into media surface No support currently for discrete key down/up events
Return value:	`true` on success, `false` on failure

Function:	`bool scrollByLines( int lines );`
Description:	Scroll display by number of lines if appropriate (for example, in a Web related media)
Return value:	`true` on success, `false` on failure

Function:	`bool focus( bool focus );`
Description:	Enable/disable focus If media Impl doesn't have focus, keyboard input will not succeed
Return value:	`true` on success, `false` on failure

Navigation

Function:	`bool navigateTo( const std::string url );`
Description:	Request navigation to given URL Typically the actual navigation happens asynchronously
Return value:	`true` on success, `false` on failure Note: success here means the navigate command succeeded. The actual navigation may fail later

Function:	`bool navigateForward();`
Description:	Navigate forward to next location in media source In a Web related media source this means go to next page in history stack In a video related media source, this means go to next chapter (currently unimplemented)
Return value:	`true` on success, `false` on failure

Function:	`bool navigateBack();`
Description:	Navigate backward to previous location in media source In a Web related media source this means go to previous page in history stack In a video related media source, this means go to previous chapter (currently unimplemented)
Return value:	`true` on success, `false` on failure Note: success here means the navigate command succeeded. The actual navigation may fail later

Function:	`bool canNavigateForward();`
Description:	Determine if it's possible to go forward to next location in media source
Return value:	`true` if it's possible to navigate forwards in the history stack, `false` if it's not possible to navigate forwards in the history stack

Function:	`bool canNavigateBack();`
Description:	Determine if it's possible to go back to previous location in media source
Return value:	`true` if it's possible to navigate backwards in the history stack, `false` if it's not possible to navigate backwards in the history stack

Caching and Cookies

Function:	`bool enableCookies( bool enable );`
Description:	Enable/disable the saving of cookies Currently only supported on a global level - all or nothing Probably only meaningful for Web related media sources
Return value:	`true` on success, `false` on failure

Function:	`bool clearCookies();`
Description:	clears any current set cookies Only meaningful for Web related media sources
Return value:	`true` on success, `false` on failure

Function:	`bool clearCache();`
Description:	Clears the cache Meaningful for Web related media May be meaningful for QuickTime media too
Return value:	`true` on success, `false` on failure

Proxy

Function:	`bool enableProxy(bool enable, std::string proxy_host_name, int proxy_port);`
Description:	Enable/disable support for network proxying. Each Impl is responsible for implementing the proxy support itself Currently only supported for LLMediaImplLLMozLib
Return value:	`true` on success, `false` on failure

Emitter/Observer

Function:	`bool addObserver( LLMediaObserver* subject );`
Description:	Add yourself as an observer of the media library See sample code for examples of how to do this
Return value:	`true` on success, `false` on failure

Observable events

Add class as an observer to the media library using addObserver and remObserver
To observe specific events, override one or more of these methods in your own code
- onMediaPreroll - emitted when the media is prerolling
- onMediaLoaded - emitted when the media is loaded
- onMediaSizeChange - emitted when the media changes size
- onMediaContentsChange - emitted when the contents of the media stream changed
- onMediaStatusTextChange - emitted when the status text associated with the media stream changed
- onNavigateBegin - emitted when navigation to a new URL starts
- onNavigateComplete - emitted when navigation to a new URL ends
- onUpdateProgress - emitted when navigation to a new URL makes progress
- onLocationChange - emitted when a URL changes - for example, an HTTP redirect
- onClickLinkHref - emitted when a link is clicked in a Web related media source
- onClickLinkNoFollow - emitted when a link that has been marked as not to follow is clicked in a Web related media source - a secondlife:// URL for example
Event payload is also passed in with data specific to the nature of the event

Examples

Example code exists in 2 places:

- In the LLMedia library itself, there are 2 example Impls called (unimaginatively) Example1 and Example2.
  - Example1 drags a random pattern on the screen and changes the contents when you move the mouse over it
  - Example2 bounces a square around the media area

- In the llmediatest application in the indra/test_apps directory.
  - Simple media browser
  - Dependency on GLUI for its UI

Current Impls

LLMediaImplExample1 - Example Impl that displays a random pattern of squares and writes data into the media surface at the mouse position
LLMediaImplExample2 - Example Impl that bounces a random colored box around within the media surface
LLMediaImplLLMozLib - provides Web based content via LLMozLib library (which, itself wraps Mozilla/Gecko 1.8.1)
LLMediaImplQuickTime - provides QuickTime media content (Movies, images, RTSP streams)