Flashing sonoffs with custom firmware

First things first.

DO NOT work on any sonoff devices that are plugged into the mains, at best you may get a nasty shock, and at worst you may kill yourself.  Do not mess with mains devices if you don't know what you're doing, or local laws prevent it.

Sonoffs are relays.  They turn things on and off.  There are many different versions available, but the good thing for home hackers is that they are based on the 8266 chip, and there are lots of clever people who've done lots of work to make replacing the factory software with your own easy.

I will show you how to flash sonoff and sonoffrf devices as that's what I've got, and I'll be adding firmware that allows the sonoff to respond to MQTT commands.

You will need

• The Arduino IDE to compile your firmware [Free]
• A firmware of your choice [Free]
• A FTDI FT232RL USB to TTL Serial UART Converter Module 5V / 3.3V Arduino, PIC
  [£2-3]
• Male pin headers 2.54mm [£1]

First a sonoff

The device will have a paper seal that shows tampering, run a 'spudger' or flat blunt knife blade in the groove at the bottom of the housing and unclip the top cover, it should click off easily.

The insides should look like this

The thing that allows us to do our magic is the unpopulated connectors on the board.  You'll need to buy some pin headers and then solder them in place

When you've finished it should look like this

I populated all 5 pins, but we only need the left 4 for our reflashing.


Next you will need your software set up.   I don't plan to reinvent the wheel, so there's a good video by Bruh Home Automation https://www.youtube.com/watch?v=-JxPWA-qxAk so have a look at that - it gives lots of good information, though I would do one or two things differently - it's a good overview with lots of details though, so well worth a watch.

To replace the original firmware in my sonoffs I use the Tasmota firmware which can be obtained from https://github.com/arendst/Sonoff-Tasmota

Download the zip file, extract it and put it in the path of your Arduino IDE software.

This should be something like

path/Arduino/Sonoff-Tasmota-versionnumber/project/

where project is something like sonoff1

You will have to make some changes to the code in order to customise it to your wifi and needs - this is covered in the Bruh HA video above, but a list of changes is as follows.

In your

path/Arduino/libraries/PubSubClient/src/

folder find and edit PubSubClient.h

Comment out the line that specifies the max packet size by putting // at the beginning of the line
Copy that line and paste it afterwards and change the number to 512
It should look like below


//#define MQTT_MAX_PACKET_SIZE 128
#define MQTT_MAX_PACKET_SIZE 512


Save PubSubClient.h



The only other file you need to modify is user_config.h
This can be found in

path/Arduino/Sonoff-Tasmota-versionnumber/project/



Change the following lines


#define PROJECT                "sonoff1"


sonoff1 will be the name of your switch when you use it, so you could call it something more specific like kitchenswitch1 etc

Next update some wifi settings

#define WIFI_IP_ADDRESS        "0.0.0.0"       
#define WIFI_GATEWAY           "192.168.1.1" 
#define WIFI_SUBNETMASK        "255.255.255.0" 
#define WIFI_DNS               "192.168.1.1" 


Change 192.168.1.1 to whatever you use on your lan


Now enter your wifi name and password

#define STA_SSID1              "Wifiname"               // [Ssid1] Wifi SSID
#define STA_PASS1              "secretpassword"     //

You can also add

#define STA_SSID2              "alternativewifiname"      // [Ssid2] Optional alternate AP Wifi SSID
#define STA_PASS2             "alternativepassword"

if you've more than one wifi network avaiable.


I run my own mqtt broker on my RPi

So we need to configure that

#define MQTT_HOST            "192.168.1.10"          // [MqttHost]
#define MQTT_PORT            1883              // [MqttPort] MQTT port (10123 on CloudMQTT)
#define MQTT_USER            "brokeruser"       // [MqttUser] Optional user
#define MQTT_PASS            "brokerpassword"       // [MqttPassword] Optional password


Change the details to those that you use.


Next it's possible to send a command to all sonoff devices eg turn everything off

You may set up all your kitchen devices to respond, or as I'm doing set all sonoffs to respond to this group topic (rather than telling each switch specifying to turn on or off)

#define MQTT_GRPTOPIC          "sonoffs"

and finally, give your device a friendly name

#define FRIENDLY_NAME        "Sonoff1"


There are other things you can configure like security etc but I'm not covering that.

Once you've done all that you're now ready to flash.

Check the settings on your IDE are as follows

• Tools->Board->Generic ESP8266
• Flash mode ->DIO  [Check firmware version as later ones say DOUT should be used]
• Flash frequency -> 40MHz
• CPU Frequency-> 80MHz
• Flash Size-> 1M (64k SPIFFS)
• Debug port-> Disabled
• Debug level-> None
• Reset Method->ck
• Upload Speed-> 115200
• Port->Where you connected your FTDI board (eg Com5 or /ttyUSB0 etc)

Press the compile button on the IDE, if all goes well it will tell you.  If not, look at the error messages - it will most likely be a typo, or you need to install a library that the code uses, but you don't have.

Tools->Add files->Manage libraries

Will allow you to search for and add any libraries you need.

Assuming you've no errors

Connect your sonoff to your FTDI as below, plug the usb lead into your PC at this stage.  Make sure you FTDI is set to 3.3v

I've had a fair bit of frustration at this point, but the most reliable way to get the firmware uploaded is as follows.


Plug in the usb to your computer, wait until the port for the FDTI device is selectable in the Arduino IDE, then select it.

You will see the FDTI connectors from left to right are

VCC, Tx, Rx, Gnd  the Gnd IS NOT connected at this point.

We need to set the sonoff to accept new firmware, in the case of the basic sonoff what is needed is to
PRESS and HOLD DOWN the on/off switch, while AT THE SAME TIME connecting the Gnd to the 4th pin (the one next to the green wire in the picture above, the 5th pin is not used).

Once you've slide the Gnd wire on the 4th pin you can release the switch.

If when you're doing this you break the connection between the ground wire, and the ground pin then you will have to start again, as the sonoff will not be in a mode to accept new firmware.

When you've done this you can now press the upload button in your IDE.  Your FDTI board led will flash and you should see lots of dots in the IDE indicating progress.

Hopefully this reaches 100% in which case you're done.

Your sonoff should now be able to connect to your MQTT broker and respond to commands to /sonoffs/sonoff1

If you receive errors then try again.  I had most problems when using a 5v logic level 8266 based plug, and think the voltage was boarderline, so moved the USB on the computer to a port directly attached to the board, rather than one on the front plate of my PC, and that allowed successful flashing.

The process for flashing other sonoff devices is similar, though sometimes a bit more awkward.

I've also got sonoffrf devices

Outwardly they look similar, but inside they've a daughter board with the Rf receiver, but the same empty pin holes on the main board.

Solder pin headers as before so you can attach your FTDI board

The process for flashing a sonoff Rf is exactly the same as the basic sonoff, with one difference.

In order to put the sonoff Rf into a mode that allows firmware update we DO NOT press the button down when connecting the ground wire.  Instead it's a bit more fiddly.

We need to look at the underside of the board

And then instead of holding down the switch we need to short these two points when connecting the ground wire.

This is fiddly, some have had success doing this with the leads of an led, I used a pair of metal tweezers. If you have difficulty, or you'll be doing this a lot, it may be worth rigging up an inline switch for the last FDTI wire so you can make the connection more easily when shorting these two points.

The rest of the process is the same as before.

Now that you've one (or more) sonoffs flashed, the next article will be on controlling them with node-red and an MQTT broker.

For subsequent sonoffs, you would only need to change the

#define PROJECT                "sonoff1"

and

#define FRIENDLY_NAME        "Sonoff1"

so that each sonoff had it's own unique identifier.


I should add one more thing, sonoffs don't have an earth wire, but your device may need one.  If that's the case the you should have an earth wire that bypasses the sonoff.  If that's the case then I would suggest that you also place your sonoff in an enclosure - I'd actually always recommend this.  That also has the advantage that you'll be able to add additional cable strain relief to the sonoff wires, no bad thing as you don't want it to be possible to pull a live mains wire free from the end of your device.

See https://www.youtube.com/watch?v=8mz5sCAvDAY for some safety advice




It's possible to apply OTA updates to sonoffs, or have them check an appstore for new firmwares, but that's not covered here.

My small solar power system

This is the first part of a series of blog posts that will cover how I've set up a small solar electricity system, that generates and stores solar energy, and how I monitor and control it.  

I should be very clear at the outset, I'm no expert, and am not qualified in this field - I've just read a bit and have built my own system.  I take precautions as best I can, but if you're contemplating doing anything like this yourself, make sure you know what you're doing as you can get a shock that could hurt you, or especially if you're using an inverter to generate mains electricity, you could get at shock sufficient to kill you.

I'm doing this small scale - it's a hobby, and I generate some electricity that I use to power all my usb devices (phones, tablets and cameras) all summer.  I also store the electricity and power my living room TV.

It's the sort of system that would be useful in a caravan/camper van or on a boat.

So what have I got ?

2 x 100 Watt Panels	2 x 250 Watt Panels
2 x Epsolar Tracer A (30A)
Approx. 200Ah Lead Acid Batteries (24V configuration)	Approx 3kWh Lipo powerwall (7S configuration)
A busbar for connecting storage and chargers	A Big Red Button for killing the Load
A number of sonoff switches	A raspberry Pi

I used to have more lead acid batteries but I'm slowly transitioning away from them, and my system was originally a 12V one, but now is configured as a 24V system.

I've been building a 3kWh powerwall with lithium polymer batteries - some are reused laptop cells, and others are new ones.


By themselves, the solar charge controllers can do an awful lot, but with the addition of a small computer such as a RPi and sonoffs they become much more flexible.

Elsewhere in this blog I've already covered how to add software to the RPi to obtain, store and display the data from your EPSolar Tracer A charge controllers.


To display what your system is doing and that gives output like this

What I'll be covering in the next series of posts is more to do with what you can measure and control once you've got this data, and a little more hardware, and how you can integrate your solar data with other sensors and actuators.

I'll be using node-red to do this, it's a simple interface for controlling things.  We'll start off by making small flows, but once you get going the complexity of your control can build to meet your needs.  It produces really crisp and clean dashboards, controls and graphs, you can make small ones for phones

Or larger dashboards for PC or tablets

But you can also leave node-red running in the background, and have it monitor, and turn things on and off as necessary without any input from you.

Emailing you a daily solar performance update

The ability to see the live data, and view a performance graph is great, but sometimes you'll be away from home and be wondering how your system is doing.

You could spend some time configuring your network to allow remote connections by changing some firewall rules and adding some authentication, or vpn if your router supported it - that would work well, and would mean that you had all the abilities that you have when you're at home.

However there is an alternative method, in reality it's probably sufficient for you to get a daily email of the performance data sent from your Raspberry Pi - that would have the advantage that since that traffic is outbound, it would require no firewall changes, and you could pick up your email anywhere - so that's the method I'm going to describe now.

These instructions presume you've already got a Raspberry Pi or similar making periodic connections to your charge controller and you can read the data on demand.

All that's needed in addition to that is -

• Some software to send emails (smtp client)
• A couple of cron jobs to send the emails and tidy up old data
• A gmail account

Create a Gmail account 
You could use an existing one, though for security you may just make a new one, as you may have to configure it to use less secure authentication and the username and password are stored in plain text on the Pi, which, depending on your setup, may be in an relatively insecure location, e.g. garden shed.

Test the Gmail account to make sure you can send an email and have the email address and password to hand.


Install an smtp client for php 
Full instructions for this are given at
https://www.digitalocean.com/community/tutorials/how-to-use-gmail-or-yahoo-with-php-mail-function

If you're on Raspbian or similar follow the Debian/Ubuntu instructions.


Make some cron jobs
I send my daily email at 9am - I chose this time as at most times of the year there's light at that time of day, and I've found that to be necessary in order to be able to get the kWh data from the charge controller reliably with my setup.


I have the following root cronjobs


* * * * * /var/www/html/epsolar/getsolarstats.php

This harvests performance data every minute and stores it in a database - this is for the dashboard.


50 8 * * * truncate -s 0 /home/pi/report.txt

At 08:50 each day I delete the contents of the previous days email.

 0 9 * * * (sleep 30; /var/www/html/epsolar/example_cli.php >> /home/pi/report.txt)

At 09:00 I run the command line solar data harvesting script and save it to a text file (report.txt).  If this were done exactly at 09:00 then it would clash with the script that runs every minute to do similar things, and would end up not getting data.  Cronjobs have a resolution of 1 minute, so without a little trick there would always be a clash whatever time I chose to do this.

The trick to avoid the clash is to trigger the cron job, but to include a sleep 30 command which makes the command wait for 30 seconds then do it's data gathering - in this way the clash is avoided and both scripts get their data.


As the Pi user I have the following cron job

1 9 * * * cat /home/pi/report.txt | msmtp -a gmail targetemailaddress

Send the email at 09:01


Script modifications
I modified /var/www/html/epsolar/example_cli.php to include

#!/usr/bin/php

as the first line and

$tracer = new PhpEpsolarTracer('/dev/ttyUSB21');

contains the device connection parameters (yours will differ).


The original comes from https://github.com/toggio/PhpEpsolarTracer

I further modified it to order the data into the desired sequence, and I removed some sections that I didn't need.


That's it - I now get a daily email so I can keep an eye on my voltages and power generation when away from home.