Yuku free message boards

Things are still changing rapidly on the BBC website.

With regard to Part A of my post earlier this month (#3, above)  the high quality MP4 streams (actually MP4 Audio [M4A] in FLV), step 5 of the instructions I posted is already out of date. The new step 5 (actually steps 5, 6 and 7) is as follows -


5. On the page you opened at step 4, find the section marked "aac" and "bitrate=128". The active parts of that section, the black text within the quotation marks, can be combined to create the download link.

The order in which those parts go together is this:

rtmpt://server/application/identifier?authString

The two most important items are those marked identifier and authString, so find them on the page you opened. You'll need them both in the next step.

6. The assembled download link looks like the following example; you must replace the blue section in it with the text contained in the identifier item (step 5 above) and replace the green section with the text contained in the authString item (step 5 above), but you must NOT delete any of the text shown in red:

rtmpdump -r "rtmpt://bbcmedia.fcod.llnwd.net/a1414/e3/mp4:bbc7/
secure_auth/RBN2_bbc7_-_thursday_0130_b00sbtkn_2010_05_13_01_38_32?
s=1273729782&e=1273744182&h=601641381aebf71a9d0702160a9d9f62"
-o filename.flv

Note: The example has wrapped around and become 4 lines, so as to fit on this page. But in order to work the completed command MUST be all on one line.

Build the command line by doing a copy-and-paste. Copy each text section you need, in the order you need them (from above, and from the BBC page you opened at step 4), and paste each in turn into a .BAT batch file (a plain text file with the extension .BAT).

7. Put the batch file in the same directory (folder) as the program RTMPDUMP.EXE; then open an Internet connection; then run the batch file (i.e. double-click on it).

Note: The command line will be valid for a couple of hours, but won't work the following day because the authString text in the BBC page you opened at step 4 will have changed by then. But you can open that page again, and get the new authString text.


The above instructions will download a 34MB .FLV flash file, containing an MP4A (MP4 Audio) file encoded in AAC audio format, at a sampling rate of 44.1 kHz, with an average bitrate of 128 kbps.

The FLV file plays in GOM Player v2.1.21.4846 (or here), and in FLV Player v2.0. For Windows 9x, I recommend GOM Player (a superb audio and video player).


If you'd prefer a smaller file, in step 5 find the section marked "aac" and "bitrate=48" instead. But in all other respects the procedure is the same.

Here is the latest: a new method for recording Flash RTMP files from the BBC iPlayer.

The following is optimised for Windows 9x, but will also work in Windows XP.

The following instructions enable you to find the URL address of an RTMP stream, i.e. the real address of a BBC iPlayer radio or tv show, using a program called RTMPSRV.EXE, and you then use that address with the main program (RTMPDUMP.EXE) to download that show:-

1. Use Get_iPlayer v2.76 to download the RTMP stream info for the BBC radio or tv show which you want to record, by inserting the 8-digit PID (Programme ID) number from the URL address of the iPlayer page for that show in place of the 8 zero's in the following command -

perl get_iplayer.pl --type=radio --streaminfo PID:00000000 > BBC.txt

(As it happens, this new Get_iPlayer v2.76 also gives Windows XP users a means of recording the 128kbps MP4-in-FLV high quality stream by combining Get_iPlayer v2.76 with RTMPdump v2.2d using the commands which worked with Phil's original Get_iPlayer. But that's another matter entirely.)

There are two lines of code which you need to change in the file get_iplayer.pl in order for this new Get_iPlayer 2.76 to work with the altered BBC iPlayer. In the section that's headed ‘# Limelight CDN’ change the following two lines of get_iplayer.pl, using Notepad.exe:

Change $conn->{playpath} = "$cattribs->{identifier}?$cattribs->{authString}"; to:           $conn->{playpath} = "$cattribs->{identifier}"; Change $conn->{application} = $cattribs->{application}; to:           $conn->{application} = "$cattribs->{application}?$cattribs->{authString}";

You will also need to make one other change. To prevent get_iplayer.pl "phoning home" to a site that no longer exists, you need to comment out the 7-line section in it that hunts for plugin updates ( by putting a # mark at the start of each line ). To find that section, just search the get_iplayer.pl file - using Notepad.exe - for the phrase 'plugins update'.

Finally, you will also need to copy the file rtmpdump.exe into the drive C: root directory


2. All you need from the text file which the previous step creates is the HOST address, so that step is unnecessary 99% of the time, because for the BBC iPlayer the host address is normally this:

    bbcmedia.fcod.llnwd.net


3. Edit your computer's HOSTS file, as follows:

(a) Go to START > RUN and type this in the box, then click OK:

     C:\WINDOWS\COMMAND\ATTRIB.EXE -R -S -H C:\WINDOWS\HOSTS

(b) Go to START > RUN and type this in the box, then click OK:

     EDIT C:\WINDOWS\HOSTS

(c) Disable the line in the HOSTS file that reads
"127.0.0.1 localhost" by putting a # at the start of that line

(d) Type in the following line, immediately underneath that line:

     127.0.0.1 bbcmedia.fcod.llnwd.net

(e) Save the changes: File > Save

NB: When it's all over, put your HOSTS file back to normal by reversing all these changes.


4. Download the program RTMPDUMP version 2.1d from this link:
http://rtmpdump.mplayerhq.hu/download/rtmpdump-2.1d-windows.zip

The .zip file includes RTMPSRV.exe as well as RTMPDUMP.exe, so run RTMPSRV.exe (by double-clicking it). RTMPSRV and RTMPDUMP are DOS applications, so this will open a Command window (also known as a DOS Box or Command Prompt window).

Note: RTMPdump v2.1d is the last version that can run on Windows 98/98SE/ME. (If you are running Windows XP, you'll need to get RTMPdump v2.2d or later instead.)


5. Open the iPlayer page that plays the radio or tv show you want to download, and start it playing. This is an example of the page URL address (you must substitute the show's 8-digit PID number for the 8 zero's in this example) -

    http://www.bbc.co.uk/iplayer/console/00000000


6. Now look at the window in which RTMPSRV is running. It will start displaying the true RTMP address of the tv or radio show that the iPlayer is trying to play in the other window.


Command Format:

rtmpdump -r "rtmp://[server]:[port]/[application]?[authString]"
-a "[application]?[authString]" -f "[Flash Version]" -W "[swfUrl]"
-p "[iPlayer page URL]" -y "[mp4: to .mp4 playpath]" -o file.flv


Command Example:

rtmpdump -r "rtmp://bbcmedia.fcod.llnwd.net:1935/a1414/e3?as=adobe-hmac-sha256&av=1&te=connect&mp=bbc7/secure_auth/RBN2_bbc_-_thursday_1930_b00sg0w7_2010_05_20_19_26_38.mp4&et=1274934408&fmta-token=2b4b12633bfd205b3d7f3060c0f827c37ae18080883697444853b3c7866b40ee" -a "a1414/e3?as=adobe-hmac-sha256&av=1&te=connect&mp=bbc7/secure_auth/RBN2_bbc_-_thursday_1930_b00sg0w7_2010_05_20_19_26_38.mp4&et=1274934408&fmta-token=2b4b12633bfd205b3d7f3060c0f827c37ae18080883697444853b3c7866b40ee" -f "WIN 9,0,260,0" -W "http://www.bbc.co.uk/emp/9player.swf?revision=18182_20020" -p "http://www.bbc.co.uk/iplayer/console/b00sg1gl" -y "mp4:bbc7/secure_auth/RBN2_bbc7_-_thursday_1930_b00sg0w7_2010_05_20_19_26_38.mp4" -o filename.flv


7. Write a batch (.BAT) file called RTMP_SRV.BAT to run RTMPSRV.EXE and pipe its output into a second .BAT file, one that can be used then to actually download the iPlayer show -

This is the content of RTMP_SRV.BAT:-

    rtmpsrv > rtmpdump_run.bat

If you run RTMPSRV.EXE by double-clicking on the batch file RTMP_SRV.BAT, it will save all its output into RTMPDUMP_run.bat - but remember: it is saving the URL address of the iPlayer show, not the actual show itself!

Note: You can NOT run RTMPDUMP_run.bat to download the iPlayer show until you have re-edited your HOSTS file back to its original state (or you'll get an error message).

Note: There will be many copies of the download command saved into the batch file RTMPDUMP_run.bat so you must delete all except one! Otherwise the batch file will keep deleting (i.e. overwriting) the download every time it completes!


8. Finally, you must change the file extension of the downloaded
file from .MP4 to .FLV because it is really a Flash FLV file!

A Flash audio or video file will play in GOM Player or in FLV Player v2.0.

There are tools to extract the MP4 file from inside the FLV wrapper, if you want to do that, e.g. FFMPEG.EXE. Here is the FFMPEG command:-

    ffmpeg.exe -i "input.flv" -acodec copy "output.mp4"

That command does not transcode the file, but makes a simple copy of the mp4 audio data (this is the function of the 'copy' option), essentially just creating a new file that omits the original FLV wrapper. You are left with a regular MP4 Audio file, containing the original aac format audio that was in the FLV file.

An MP4 Audio file will play in Winamp or in Media Player Classic or in Windows Media Player (in addition to playing in the excellent GOM Player).

Remove this ad

Roddo: I'm delighted that WM Recorder v14 (for Windows XP) can apparantly record the Macromedia Flash stream (the 128 kbps stream).

Be careful, though, to ensure that it is actually receiving the RTMP flash stream: the BBC also streams all of its radio shows in lower-quality WMA audio (a 96 kbps stream), and as you are using a WMA recorder it's obviously capable of recording those WMA streams.

Unfortunately, I can't offer any suggestions. As I don't use Windows XP, I can't run any tests with WM Recorder.

In general, since WM Recorder only runs on Windows XP/Vista, and is not available for Windows 9x, Mac or Linux, I still suggest RTMPdump v2.1d is the best solution for those operating systems in downloading radio shows from the BBC's "listen again" service.

Note: RTMPdump v2.1d is the last version that can run on Windows 98/98SE/ME. (If you are running Windows XP you'll need to get RTMPdump v2.2d or later.)


Incidentally, all RealAudio streams (i.e. RTSP streams) for BBC National Radio were switched off on May 27th, completing the BBC's switch-over from RealAudio to Macromedia Flash.

For this week only (i.e. until Thursday, June 3rd) some RealAudio links on the BBC website will continue to deliver "listen again" podcasts in RealAudio (as well as the Flash and WMA versions), in respect of radio shows which were broadcast during the week prior to May 27th.

RTMPdump.exe issues a request to the RTMP server for a connection, by sending a command including the requisit elements, thereby initiating the process of negotiating a connection. Such a command may be constructed using connection and authentication information obtained previously from the server by client software applications such as rtmpsrv.exe

In negotiating a connection, an RTMP server sends and receives a data stream containing multiple elements, as a single command sequence. An on-demand stream, using protocol 0 (not tunnelled, not encrypted), typically includes the following elements -


A. For a Limelight server

--protocol : Protocol types include 0 (RTMP), 1 (RTMPT) [Tunnelled], or 3 (RTMPE) [Encrypted]
--host : server address (a.k.a. domain address)
-a : authentication elements (the alternative --app may be used instead)
Typically in the format -
{application}?as={data}&av={data}&te={data}&mp={data}&et={data}&fmta-token={data}
application: A path address. For example, a1414/e3
as : For example, as=adobe-hmac-sha256
av : For example, av=1
te : For example, te=connect
mp : Typically, two or more comma-separated URL addresses, for alternative bitrate streams (mpeg format, mp3 or mp4)
et : Typically, a ten-character number (numerical)
fmta-token : Typically, a 64-character authentication (auth) string [i.e. an authentication token] (alphanumeric)
-y : playpath (URL address of the desired bitstream, one of those specified in mp above)
Typically, in the format mp3:{path}/{filename}.mp3 or mp4:{path}/{filename}.mp4
-o : Output filename


The foregoing are typically the only elements (or "switches") that are essential to a connection, if neither Tunnelling nor Encryption are in use by the server. Although other elements may be encountered in practice, they are normally non-essential.

Hence the following elements are typically sent by RTMPdump, as a single command line -

rtmpdump --protocol 0 --host xxxxxxxx.fcod.llnwd.net -a path?as=data&av=data&te=data&mp=data&et=data&fmta-token=data -y mp4:URL/filename.mp4 -o file_mp4.flv


The parts comprising the -a (or --app) element must be incorporated in it in the order shown above, as the sequence in which its parts are received by the RTMP server is critical.

The authentication strings (et= and fmta-token=) contain session information, so will change on each fresh connection made to the server (which in practice typically means they will expire if a new session is begun, not literally on every attempt to resume a connection), but the other elements will not usually vary from session to session.


B. For an Akamai server

The command sequence is typically as above, except that the -a (authentication) string contains the following elements instead -
auth : Typically, a 62-character authentication (auth) string [i.e. an authentication token] (alphanumeric)
aifp : For example, aifp=v001
slist : Typically, the URL address of the stream

Hence the following elements are typically sent by RTMPdump.exe, as a single command line -

rtmpdump --protocol 0 --host xxxxxxx.edgefcs.net -a ondemand?auth=data&aifp=data&slist=data -y mp3:URL/filename -o file_mp3.flv


The parts comprising the -a (or --app) element must be incorporated in it in the order shown above, as the sequence in which its parts are received by the RTMP server is critical.

The authentication string (auth=) contains session information, so will change on each fresh connection made to the server (typically, if a new session is begun, e.g. the computer is restarted, not literally on every attempt to resume a connection). The other elements will not usually vary from session to session.


Note - The above describes the simplified form, whereby the stream is first saved to the user's hard disk, to be played back thereafter in a media player capable of playing an FLV encoded file (H.263 or H.264 encoding), such as GOM Player. If it's desired, instead, to play the stream directly from the RTMP server, thus giving immediate playback, then additional elements will be needed in the command string including -

-f : This specifies the version of the Flash plugin installed on the user's computer. For example, -f "WIN 9,0,260,0" [which indicates the user has the Windows version of Flash Player 9, release 260]

-W : The capital W command. This is the URL address of the SWF player used to play the stream, as indicated by the web page from which the stream is derived. For example, path/9player.swf?revision=18269_21576



EXAMPLES from the RADIO iPLAYER

Here are two examples, from the BBC's Radio iPlayer. Both of these are for an on-demand ("Listen Again") file. These examples can't be used with a Live stream.

[Note: For a tv show, the principles are similar (but the practice is more difficult, because there are often as many as five or six different bitrate streams to deal with at every step, and the download sizes are huge: up to 800 MB for a 50 minute tv show, compared to 30 MB for a half-hour radio show).]

First, open the Playlist page for the radio show you want to download (replace the 8 zero's with the 8 digit ID number from the URL of the page that plays the radio show in question):-

http://www.bbc.co.uk/iplayer/playlist/00000000

Second, open the Media Selector page for that radio show (replace the 8 zero's with the 8 digit Identifier from the Playlist page), as the Media Selector page contains all the information needed below:-

http://www.bbc.co.uk/mediaselector/4/mtis/stream/00000000



A. The highest quality on-demand stream: AAC in MP4A in FLV -

This is the command string suggested by rtmpsrv.exe -

rtmpdump -r "rtmp://bbcmedia.fcod.llnwd.net:1935/a1414/e3?as=adobe-hmac-sha256&av=1&te=connect&mp=bbc7/secure_auth/modem/RBN2_bbc7_-_friday_2300_b00bf9yc_2010_09_10_22_55_13.mp4,bbc7/secure_auth/RBN2_bbc7_-_friday_2300_b00bf9yc_2010_09_10_22_54_46.mp4&et=1284234325&fmta-token=65d90a5b552fdded243f9a43bc2cebd4c9dd134c7b0a8970d646ebb8e3cd59e0" -a "a1414/e3?as=adobe-hmac-sha256&av=1&te=connect&mp=bbc7/secure_auth/modem/RBN2_bbc7_-_friday_2300_b00bf9yc_2010_09_10_22_55_13.mp4,bbc7/secure_auth/RBN2_bbc7_-_friday_2300_b00bf9yc_2010_09_10_22_54_46.mp4&et=1284234325&fmta-token=65d90a5b552fdded243f9a43bc2cebd4c9dd134c7b0a8970d646ebb8e3cd59e0" -f "WIN 9,0,260,0" -W "http://www.bbc.co.uk/emp/9player.swf?revision=18269_21576" -p "http://www.bbc.co.uk/iplayer/console/b00bf9z0/Old_Harrys_Game_Series_4_Episode_2" -y "mp4:bbc7/secure_auth/RBN2_bbc7_-_friday_2300_b00bf9yc_2010_09_10_22_54_46.mp4" -o RBN2_bbc7_-_friday_2300_b00bf9yc_2010_09_10_22_54_46.mp4


The above format works. But here is the simplified (i.e. more useful) format from my .BAT batch file -

SET mp=bbc7/secure_auth/modem/RBN2_bbc7_-_wednesday_1300_b006v244_2010_08_25_13_00_33.mp4,bbc7/secure_auth/RBN2_bbc7_-_wednesday_1300_b006v244_2010_08_25_13_00_06.mp4

SET et=1282760597

SET fmta-token=7d9c1938f53e65d3a30a0d9e313a917d580ec0055ecfa5b9d45bb35dc1aa7593

SET identifier=mp4:bbc7/secure_auth/RBN2_bbc7_-_wednesday_1300_b006v244_2010_08_25_13_00_33.mp4

rtmpdump --protocol 0 --host bbcmedia.fcod.llnwd.net -a a1414/e3?as=adobe-hmac-sha256&av=1&te=connect&mp=%mp%&et=%et%&fmta-token=%fmta-token% -y %identifier% -o output_mp4a.flv



B. The MP3 on-demand stream: MP3 in FLV -

This is the format from my .BAT batch file -

set server=cp48188.edgefcs.net
set auth=daEdwabaccObnascHdZbCb4d7cxcRanc5bf-bmI9Jl-bWG-DnmDCpwoIBvGqzD
set aifp=v001
set slist=secure/bbc7/RBN2_bbc7_-_saturday_1000_b005yg5q_2010_09_11_12_36_00
set identifier=mp3:secure/bbc7/RBN2_bbc7_-_saturday_1000_b005yg5q_2010_09_11_12_36_00

rtmpdump --protocol 0 --host %server% -a ondemand?auth=%auth%&aifp=%aifp%&slist=%slist% -y %identifier% -o output_mp3.flv

Playing a Live Stream while simultaneously saving it

Here's my solution for playing a live stream whilst rtmpdump is saving it to disk, or for playing an on-demand stream whilst rtmpdump is downloading it, using Windows 98.

Note - The following solution will work for ANY download of a radio or television show (even non-BBC shows), using ANY download method (not just rtmpdump). Even if you use an automated downloader such as get_iplayer or iplayer_dl the following procedure will play the file while it's still downloading - a godsend for live streams!)

This uses the program FFPLAY.EXE (which is included in standard distributions of ffmpeg). You run it in a seperate (i.e. second) DOS window at the same time that rtmpdump is downloading the rtmp stream in a DOS window.

Here's a version of ffmpeg which will run under Windows 98 (SVN-r16573) (13 Jan 2009): http://ffmpeg.arrozcru.org/builds/shared/ffmpeg-r16537-gpl-shared-win32.tar.bz2

(Note: The latest ffmpeg version which runs on Win9x is release r21874)

An important point is that the batch file must be in the same folder (directory) into which rtmpdump is saving the rtmp stream.

Another important point is that both RTMPDUMP and FFPLAY must be set to use .FLV as the file type, because the batch file (see below) sets FFPLAY to play any file with that file type in the current directory.


1. Playing a Radio show while downloading it

This is the content of my successful batch (.BAT) file for a radio show (using the option -vn to specify no video, as this is a radio stream) -

"C:\Program Files\ffmpeg\ffplay.exe" -vn *.flv

(Note: This works for both a Live stream and an On-demand stream)


Here's an alternative command line -

Play a radio stream as it downloads (with buffer, no display):

"C:\Program Files\ffmpeg\ffplay.exe" -vn -nodisp -bufsize 4096 *.flv


2. Playing a TV show while downloading it

I then tried this procedure on a television stream, successfully. I was astonished at how the crappy performance of Flash Player on Windows 98 was magically replaced by a perfect video stream. The TV stream (an on-demand tv show) played perfectly, with no hesitating or dropped frames.

I used only one trick! I didn't start the show playing in FFPLAY until rtmpdump had been downloading the show for 15 seconds, thus building-in a 15 second buffer.

Note: I had set rtmpdump to save the TV show as a file with the extension .mp4 on this occasion. If you save a show as a .flv file, remember to alter the command line (below) from *.mp4 to *.flv instead.

It was a revelation, using the following command line in the above batch file instead of the radio command -

Play on-demand TV show with 4MB Buffer:

"C:\Program Files\ffmpeg\ffplay.exe" -bufsize 4096 *.mp4

There are some useful WMA stream addresses, downloading solutions for WMA, and much additional information besides, at -

http://myweb.tiscali.co.uk/bridip/clitheroe/otr.htm