Archives For Mozilla

On Monday, May 7th, 2012, there will be a public call to discuss the Web Cryptography API. Via the W3C public-webcrypto mailing list:

Join us next Monday 7th of May, from 19:00-20:30 UTC (3:00pm-4:30pm Boston local)

*         Zakim Bridge +1.617.761.6200, conference 27978 (“CRYPT”)

*         IRC  #crypto

Please dial in if you would like to discuss or find out the plans for this important specification.

When I found out I could choose my own title at Mozilla I was ecstatic.

Of course coming up with a good one is another matter, I thought about it for days and days…

I thought about my “newbie” place at Mozilla and dredged my unconscious memory of all of the movies I had ever seen and chose the title: “Civilian Observer“.

Folks who have seen my business card are aghast – “What the hell does that mean?”

Surely you remember the 1980 movie “The Final Countdown” starring none other than Martin Sheen? (For my millennial readers, that is Charlie Sheen’s old man). His role was that of “Civilian Observer” on board the USS Nimitz.

From Wikipedia:

In 1980, the supercarrier USS Nimitz (CVN 68) takes on a civilian observer, Warren Lasky (Martin Sheen), at the orders of his reclusive and mysterious employer, Mr. Tideman (who helped design much of the ship), just before it departs Pearl Harbor for a training mission in the Pacific Ocean. Out in the Pacific, the ship encounters a strange storm-like vortex which disappears after the ship passes through it.

Naturally, I am not going to spoil it for you, but the Civilian Observer helps try and figure out alternative ideas for the strange place the ship finds itself in.

In working on open source or free software, you know there is a lot at stake. I feel like all of us are observing the growth of the internet and with it the growth of attempts at curtailing the power that this computer network gives us. We are seeing more and more of the growth of surveillance technology, some of which is created in Silicon Valley – technology that is used to prevent and record private communication, help identify and  round up and torture or kill dissidents all over the world.

The battle for the Web and the Internet is a full on war. It will always be that way. The issues of privacy vs. surveillance, open or closed software stacks and walled gardens are here to stay. We must be vigilant and we must create tools that fight and overturn the impulse to control or subordinate and use people and networks of people.

I have felt like the “Civilian Observer” since before I started at Mozilla, its just become more pronounced in the past few years. Take for instance, a smattering of my Twitter feed – tweets and links selected in the past day, mere hours:

a tweet

a tweet

a tweet

a tweet

a tweet - no link

a tweet

a tweet

a tweet

a tweet

a tweet

a tweet

a tweet

a tweet

a tweeta tweet

Sometimes these tweets pop up right after one another:

2 tweets

a tweet

Wow!, right?

Recently, WikiLeaks published a database of surveillance companies that produce tools that provide “Mass interception of entire populations…“.

These tools deployed on a mass scale essentially turn the Internet into a surveillance system.

We need more Countermeasures for this.

Firefox is one of these countermeasures, without it we would really be in a world of pain. I cannot even imagine how craptastic the net would be without Firefox.

Mozilla’s Boot2Gecko project is also a countermeasure. If we pull this off, it will truly be a Coup d’état in the mobile device space. The goal: building mobile phone apps from HTML, JavaScript and CSS! That is the way it should be.

Deuxdrop is another project in Mozilla labs trying to create a secure messaging system – I have high hopes for it. Again, these are the kinds of tools that need more focus.

I have been working on DOMCrypt for a few years now. DOMCrypt provides a Cryptography API in web pages, making it trivial and fast to encrypt data that may be part of a message to another web user or data that should stay private while stored in LocalStorage (amoung many other use cases). I am proud to say we have implementation bugs and plans lined up for both Gecko and WebKit. The W3C is using the API as the strawman proposal for the Web Cryptography Working Group. Web developers will be able to use strong and fast crypto via a DOM API. Exciting stuff.

Web developers are way out in front on this issue – there are several crypto libraries for JavaScript out in the wild, and developers are using them to push the envelop in web apps.

This is great stuff, but we need safer, built-in crypto APIs for browsers to provide a foolproof way to use crypto in the browser, not to mention the speed boost you get when calling native code APIs vs. native JavaScript functions. The other thing we need are new ways to communicate. The Web has won as the default communication channel and developers need to be able to write apps that allow people to communicate without sharing the conversation with a 3rd party.

Not sharing data with a 3rd party is the key issue. This concept destroys a lot of business models, well, one business model – the one everyone seems to think will work long term. I doubt that. Here’s a business model you can try: make a cool product and charge people a small fee to use it while simultaneously preserving their privacy. Novel, isn’t it?

What can you do to help? You can demand privacy and security be a feature of the products you use, not an afterthought. You can help test or help develop “countermeasure” applications.

Wikipedia lists several applications that enhance communications with more security:

(The link above does not mention  Tahoe-LAFS – please comment if you know of any more notable applications or toolkits)

I hope the future will bring a slew applications and startups that trumpet privacy and security enhanced applications. I hope web developers begin to think about more creative uses of their talents than online coupons and advertising-funded social media. The future of online privacy and security is both bright and bleak. Bright in that we have the CPU power, talent and base algorithms sitting right in front of us. It may seem bleak if you think no one cares about privacy or there is no ‘free lunch/free beer’ business model.

Another thing you can do is support organizations that are “watching the watchers”, the EFF, Privacy International and others. A great resource for this is

The Final Countdown Poster

The Final Countdown Poster

MozCamp Asia Report

2011/11/28 — 1 Comment

I recently attended MozCamp Asia in Kuala Lumpur, Malaysia. It was a pretty fun and amazing experience. Meeting Mozilla contributors from all over Asia was of course, the highlight of the trip.

I did a talk at the conference called “From Web Developer to Firefox Hacker“.  There were a number of web developers at the conference and I wanted to convey to them that their skills are relevant to hacking on Firefox itself. Lowering the bar to hacking on Firefox is hard to do – and technical, patch-creating contributors are very important to the project. Someone who comes into the community as a newbie from a talk or workshop like this, this year, may become a rockstar next year, you never know.

The questions I kept asking myself in putting together this presentation were “How to start?”,  “Am I covering X or Y in sufficient detail?”,  “Am I scaring potential contributors with too much detail about the Mozilla process?”, and “Did I forget anything important?” There is just so much detail to cover.

A realization dawned on me while putting together these slides: our documentation is very well done at this point. I was impressed with a lot of new MDN content I linked to that did not exist when I started 3 years ago. Slowly we are chipping away at making technical contribution easier. Perhaps what we need are a few more “starting point” type documents, presentations and workshops to kick off a slew of new patches, no matter how small.

The talk went well.  A few new folks pinged me on irc in the days after, and I was invited to Taipei to give the talk as an in-depth workshop. A Chinese translation was also discussed.

Now is as good a time as ever for technical contributors to learn our process and contribute to Firefox. It may not be easy, but it is easier than ever:)

A Working Group

2011/11/04 — 2 Comments

This week I attended the W3C annual meeting known as “TPAC” in Santa Clara. I went to discuss the possible formation of a “Web Identity Working Group” to begin the process of possibly standardizing Identity APIs and protocols, of which DOMCrypt was acting as a straw man proposal for a DOM/JS Crypto API.

The short story is that “Web Identity” APIs and protocols are still very much in an R&D phase and while incredibly important, there was not much agreement between interested parties on what to begin working on.

A high-level, hard-to-muck-up, asynchronous crypto API, on the other hand, had massive support from almost all of the interested parties involved. In the end, a “Web Cryptography Working Group” is being established, and I have thrown my hat into the ring as an “editor candidate”.

One of the first tasks is to clearly define what is “in scope”, “out of scope” and what features can be considered part of a potential “road map”. The starting point for this API will have to be a bit narrow, with no UI-based features so we can establish core functionality without too much complexity.

The current charter is here: (I have a feeling this url will change soon)

This is pretty exciting stuff. I met with a whole lot of folks from Microsoft, Google, Apple, Netflix and other companies that have many potential use cases. We need to collect as many use cases as possible in order to understand the most common uses so the first iteration will provide the best capabilities. If you have a use case in mind, do not hesitate to send it to me (ddahl + at + mozilla dot com) or the w3 mailing list, (which is yet to be set up).

After reading this thought-provoking post: I thought I would enumerate some of the concerns raised and try to explain how DOMCrypt handles at least some of problems that are inherent in JavaScript cryptography.

  • DOMCrypt is not a low-level crypto API. DOMCrypt is very high-level, making it much harder to ‘do it wrong’. The only configuration is telling DOMCrypt what algorithm to use. See:
  • Private key material is never exposed to content JS. Each keypair is represented by an ID. This keeps the key material in a secure key store outside of content. (In the case of Firefox, the key material will be stored in NSS’ key database)
  • Each domain/origin will have its own keypair. You will have to know the ‘KeyID’ to use the domain’s keypair to encrypt data. Being able to handle secret key material in content is ‘doing it wrong’. The original DOMCrypt prototype code as well as all of the crypto libraries do this. Not good.
  • All algorithms are well designed, well tested and maintained crypto functions created by tried and true cryptographers (In the case of Firefox’s implementation we have NSS under the covers). Millions of users have depended on this underlying crypto for a decade plus. Part of the motivation for DOMCrypt is that each site might roll their own crypto implementation, or use an implementation that is out in the wild, which are likely to be subtly wrong.  By providing DOMCrypt, we can leverage all the careful scrutiny that has already gone into browser crypto implementations. We are not re-building the wheel. (Random numbers are truly random as we are calling into NSS or other system crypto depending on the browser implementation)
  • Fast. This is not a concern in the Matasano article, but I think it is a valid point. DOMCrypt calls crypto functions written in C, so all of the crypto operations are quite fast.

There are, of course, still many concerns even with a built in crypto API, but I think it is a step in the right direction. Moving all of the heavy crypto operations into the browser itself and exposing only a high-level API to consumers is the beginning. Making encryption as trustworthy as possible in a malleable content JavaScript environment will take some additional work.


*thanks to Adam Barth for input

There has been so much written and discussed online and off about privacy, user control and identity lately. This dovetails nicely with the draft spec for DOMCrypt as a proposed Crypto API for web browsers.


DOMCrypt was discussed on the WHATWG mailing list last week, see:

I have summarized the discussion here:

Out of these discussions, I have updated the Firefox feature page:

A Use-cases page was created, see:

For anyone unfamiliar with DOMCrypt, this proposal in a nutshell says:

Privacy and user control on the web is of utter importance. Tracking, unauthorized user data aggregation and personal information breaches are becoming so commonplace you see a new headline almost daily. (It seems).

We need crypto APIs in browsers to allow developers to create more secure communications tools and web applications that don’t have to implicitly trust the server.

The DOMCrypt API is a good start, and more feedback and discussion will really help round out how all of this should work – as well as how it can work in any browser that will support such an API.

This API will provide each web browser window with a ‘cipher’ property[1] that facilitates:

  • asymmetric encryption key pair generation
  • public key encryption
  • public key decryption
  • symmetric encryption
  • signature generation
  • signature verification
  • hashing
  • easy public key discovery via meta tags or an ‘addressbookentry’ tag

[1] There is a bit of discussion around adding this API to window.navigator or consolidation within window.crypto

I have created a Firefox extension that implements most of the above, and am working on an experimental patch that integrates this API into Firefox.

The project originated in an extension I wrote, the home page is here:

The source code for the extension is here:

The Mozilla bugs are here:

You can test the API by installing the extension hosted at and, and going to

The API:

window.cipher = {
 // Public Key API
 pk: {
   set algorithm(algorithm){ },
   get algorithm(){ },

  // Generate a keypair and then execute the callback function
  generateKeypair: function ( function callback( aPublicKey ) { } ) {  },

  // encrypt a plainText
  encrypt: function ( plainText, function callback (cipherMessageObject) ) {  } ) {  },

  // decrypt a cipherMessage
  decrypt: function ( cipherMessageObject, function callback ( plainText ) { } ) {  },

  // sign a message
  sign: function ( plainText, function callback ( signature ) { } ) {  },

  // verify a signature
  verify: function ( signature, plainText, function callback ( boolean ) { } ) {  },

  // get the JSON cipherAddressbook
  get addressbook() {},

  // make changes to the addressbook
  saveAddressbook: function (JSONObject, function callback ( addresssbook ) { }) {  }

  // Symmetric Crypto API
  sym: {
  get algorithm(),
  set algorithm(algorithm),

  // create a new symmetric key
  generateKey: function (function callback ( key ){ }) {  },

  // encrypt some data
  encrypt: function (plainText, key, function callback( cipherText ){ }) {  },

  // decrypt some data
  decrypt: function (cipherText, key, function callback( plainText ) { }) {  },

  // hashing
  hash: {
    SHA256: function (function callback (hash){}) {  }

I am in the process of posting this proposed spec for review by the W3C webapps list and TC39. Your feedback and / or help with this effort will be greatly appreciated.

>I did a quick DOMCrypt presentation at the All Hands last week. It went well,  a lot of helpful and smart people showed up. I am thinking about all of the improvements that need to be made in the coming weeks to make DOMCrypt better.

There are 3 obvious things to do now:

1. Namespace the API to allow for future additional APIs

2. move all heavy-lifting work to a ChromeWorker

3. DOMCrypt needs a story – an understandable message – so that Web Developers and web users can understand the possibilities and the risks of not being in control of their data. Even talking about cryptography makes people’s eyes glaze over, so the move to simplify the nomenclature is essential.

The main story is a familiar one at Mozilla: User Control. Web users should be able to control who reads what they write online.  To everyone else, the message is a garbled stream of incoherent data. This should be the default mode in online communication – default and easy.

A fourth item is to figure out how to inter-operate with  existing Crypto standards. This is a very large undertaking, so I am not sure how it will play out yet. I have a lot of reading to do. I would like to have an elegant, “webby” toolkit that is easy to use and make available ASAP. Getting bogged down in standardization may work against this. Right now, I think supporting existing standards should be a long term goal which will be achieved via future APIs.


2010/01/16 — Leave a comment

>In an effort to provide realistic data in places.sqlite, I wrote a data generator in Python which inserts many records into places, and this was a good thing.

The data is entriely made up from random strings, and you end up with urls like this: http://ffhjhfj.uwtgbz.wsc

The same for tags, etc…

In order to make things more realistic and “testable” inside xpcshell, I created a crawler using Jetpack and standard Firefox XPCOM components. I have a feeling that QA might be interested in this as, so I posted to the Jetpack Gallery

It is not configurable from the outside, but I have plans for that. I am planning on making JetCrawl surf every night for a set period to increase my collection of data work with.

I need this data in crafting a new Places Query API that is fast and well tested against a rather large collection of bookmarks and history. The urls that Jetcrawl use are taken from the Alexa Top 100, so it is a common set to boot.

Automated tests will work better with this data since it is hitting predictable urls and it is the actual places apis creating the data in the first place.

If you want to try it out, please use a new profile. You can stop it by closing the tab. There is no UI as of yet.

>The timeline work I mentioned earlier has been going well. The patches are pretty well on their way. Until this lands, I am keeping the instrumentation patch unbitrotted each week.

On the “startup” team, we use the bugzilla whiteboard to mark all startup-related bugs with “[ts]”. I wanted to propose that in the meantime before the startup patch lands, feel free to mark [ft] in the whiteboard and list any function names you want to be timed with cold/warm start, platform, etc.

I will run the [ft] query often and add the timers to the instrumentation patch(es), and update the bug with the details.

Let me know if you have any ideas or questions.

>Counting Lines

2009/05/03 — 5 Comments

>Out of curiosity, I decided to see how many lines of code are in the Mozilla central repo.

I used CLOC, which stands for “Counting Lines of Code” a fast perl program

ddahl-t500 ~ % perl ~/bin/ --exclude-dir=.hg ~/code/moz/mozilla-central/mozilla
34572 text files.
33534 unique files.
56892 files ignored. v 1.08 T=134.0 s (204.0 files/s, 45458.4 lines/s)
Language files blank comment code scale 3rd gen. equiv
C++ 3096 246458 281710 1224917 x 1.51 = 1849624.67
C 1670 166078 241676 894512 x 0.77 = 688774.24
HTML 10113 92348 19579 625324 x 1.90 = 1188115.60
Javascript 4756 156677 221426 471609 x 1.48 = 697981.32
C/C++ Header 4109 100775 310865 413259 x 1.00 = 413259.00
IDL 1250 12107 0 120206 x 3.80 = 456782.80
Bourne Shell 220 16398 20518 106269 x 3.81 = 404884.89
Assembly 100 4545 2967 53678 x 0.25 = 13419.50
XML 650 14244 8639 50284 x 1.90 = 95539.60
Perl 242 6613 11643 30146 x 4.00 = 120584.00
CSS 463 7264 12306 30108 x 1.00 = 30108.00
Python 128 4880 12547 20428 x 4.20 = 85797.60
m4 22 1873 287 15602 x 1.00 = 15602.00
Java 129 2782 5736 11613 x 1.36 = 15793.68
DTD 141 2321 3028 10117 x 1.90 = 19222.30
Teamcenter def 43 60 2 4930 x 1.00 = 4930.00
SKILL 5 79 3 2680 x 2.00 = 5360.00
make 94 1554 4818 2620 x 2.50 = 6550.00
Objective C 3 235 211 959 x 2.96 = 2838.64
DOS Batch 42 175 121 621 x 0.63 = 391.23
Bourne Again Shell 5 138 452 503 x 3.81 = 1916.43
C# 4 57 261 453 x 1.36 = 616.08
Korn Shell 4 44 249 303 x 3.81 = 1154.43
Pascal 5 56 29 295 x 0.88 = 259.60
MATLAB 2 48 0 277 x 4.00 = 1108.00
Lisp 1 30 32 256 x 1.25 = 320.00
PHP 2 75 126 248 x 3.50 = 868.00
MSBuild scripts 2 2 0 242 x 1.90 = 459.80
XSLT 8 52 36 234 x 1.90 = 444.60
lex 2 64 59 223 x 1.00 = 223.00
Visual Basic 7 30 349 143 x 2.76 = 394.68
yacc 1 15 42 79 x 1.51 = 119.29
awk 3 15 46 70 x 3.81 = 266.70
SQL 1 2 0 56 x 2.29 = 128.24
sed 4 1 0 52 x 4.00 = 208.00
Ada 1 5 0 49 x 0.52 = 25.48
C Shell 2 22 13 40 x 3.81 = 152.40
D 2 7 98 16 x 1.70 = 27.20
Expect 1 0 0 1 x 2.00 = 2.00
SUM: 27333 838129 1159874 4093422 x 1.50 = 6124253.00

So: 4 million lines of code, wow.

What about Chrome?

ddahl-t500 ~ % perl ~/bin/ --exclude-dir=.hg ~/code/cpp/home/chrome-svn/tarball/chromium/src
36331 text files.
34106 unique files.
118852 files ignored. v 1.08 T=169.0 s (100.0 files/s, 28630.1 lines/s)
Language files blank comment code scale 3rd gen. equiv
C++ 4567 219341 217686 1074656 x 1.51 = 1622730.56
C 657 76387 135868 526696 x 0.77 = 405555.92
C/C++ Header 5501 142924 330982 489098 x 1.00 = 489098.00
Perl 1463 98760 135606 270429 x 4.00 = 1081716.00
Python 1202 45766 70503 200647 x 4.20 = 842717.40
Javascript 1678 37060 71865 184861 x 1.48 = 273594.28
Bourne Shell 113 18492 22420 137002 x 3.81 = 521977.62
Assembly 4 60 44 131490 x 0.25 = 32872.50
HTML 741 4491 1244 44920 x 1.90 = 85348.00
m4 18 4361 228 37068 x 1.00 = 37068.00
Objective C 185 6036 7799 29487 x 2.96 = 87281.52
IDL 342 1666 0 13779 x 3.80 = 52360.20
XML 208 604 712 6651 x 1.90 = 12636.90
CSS 30 1319 437 6492 x 1.00 = 6492.00
make 55 852 960 5860 x 2.50 = 14650.00
Tcl/Tk 23 810 1196 5507 x 4.00 = 22028.00
yacc 4 572 171 4532 x 1.51 = 6843.32
Expect 15 4 2 2113 x 2.00 = 4226.00
MATLAB 18 222 0 1937 x 4.00 = 7748.00
XSD 3 143 1129 1587 x 1.90 = 3015.30
C# 11 182 632 1226 x 1.36 = 1667.36
DOS Batch 31 138 86 649 x 0.63 = 408.87
Teamcenter def 16 29 88 521 x 1.00 = 521.00
XSLT 2 10 28 294 x 1.90 = 558.60
Korn Shell 1 39 46 223 x 3.81 = 849.63
awk 6 6 84 211 x 3.81 = 803.91
Java 4 27 0 106 x 1.36 = 144.16
MSBuild scripts 1 0 7 100 x 1.90 = 190.00
YAML 1 0 0 84 x 0.90 = 75.60
MUMPS 1 2 0 29 x 4.21 = 122.09
sed 2 0 10 26 x 4.00 = 104.00
Ruby 1 10 7 15 x 4.20 = 63.00
D 1 3 24 13 x 1.70 = 22.10
PHP 1 0 0 3 x 3.50 = 10.50
SUM: 16906 660316 999864 3178312 x 1.77 = 5615500.34

3.1 million lines…
Looks like they are catching up on us. By one metric anyway:P

If this doesn’t seem correct let me know.