If anyone is curious the encoding that Angler is using in their SWFs is base91. The encoding was hinted at in an excellent article by Palo Alto Networks but was only identified as a function named DecodeToByteArray. Below are my notes to decode and decompress the embedded SWF.
While reviewing the data I noticed all of the bytes were valid ASCII. This usually infers base64 but the characters '@'' or '$' meant it must be a modified version it. A mistake I made after deobfuscating the ActionScript was I only cursory looked at the decoder. The code and data had the patterns of base64 and I blindly assumed it was. If it was a modified version of base64 I could reconstruct all the chars from the table. This can be done by reading each character from the data into a set. From there I would need to find the right sequence of chars. Strangely, this hackish approach lead me to the encoding.
91? As in Base91? Weird, never heard of that. A search lead me to some code written by Adrien Beraud which confirmed the ActionScript is indeed Base91.
After the data is decoded with base91 each byte is XORed. Initially the key is set to a hard coded value then the key becomes the previous byte that was encoded. Once the XOR loop is completed it is decompressed with zlib. The initial XOR key is not static. In PAN's write up the key is 91 and in mine it was 75. The key can be found with a decompiler (Trillix or JPEXS) or a disassembler (swfdump). The later can be done to extract the XOR key from the command line. swfdump -a can be used to get the assembly of the ActionScript. Searching for bitxor and pushint should provide the XOR key.
Quickly written Python code for decoding and extracting the second SWF. The key will likely need to be modified.
output.bin is the binary data extracted using swfextract. The above Python code is stored in angler-decoder.py. After running the script the decoded SWF is saved to output.bin-out.bin. Then I use xxxswf.py to verify the SWF is present.
The extracted second SWF is also obfuscated.
My "cleaned up" ActionScript