Easier Laravel DB Migrations With Zero Downtime

When Laravel is paired with a Mysql DB it can be increasingly difficult to make changes as the installation grows in popularity. While Mysql is getting better with its Online DDL there are still some limitations. And even with the latest online tools Laravel’s built-in migration scripts won’t consistently use them without specialized code. To make minimal-downtime changes easier I’ve helped create an adapter for Percona’s Online-Schema Change (PTOSC) and Mysql’s Online DDL called laravel-online-migrator (LOM).

Consider a Laravel DB migration adding a column: Schema::table('my_table', function (Blueprint $table) { $table->string('color', 64)->nullable(); }); To use PTOSC the queries have to be manually written as shell commands: pt-online-schema-change D=homestead,t=my_table,h=localhost \ --user=homestead --password=secret \ --alter "ADD color VARCHAR(64)" --execute Then it must be wrapped in a PHP function like exec, or run outside the normal Artisan migrate workflow. When done outside migrate a row must be inserted into the “migrations” table for each migration, unless Laravel’s built-in migrations will never be run.

Now with laravel-online-migrator the migration script can remain unchanged. When migrate is run the script is automatically changed from this PHP code$table->string('color', 64)->nullable(); to this command pt-online-schema-change D=homestead,t=my_table,h=localhost \ --user=homestead --password=secret \ --alter "ADD color VARCHAR(64)" --execute and the command is run.

Before executing migrations the generated commands can also be reviewed for correctness with --pretend like this php artisan migrate --pretend Pretending can be helpful when one is unsure what the adapter will do. When using PTOSC that output can also be copied and pasted into a shell with the --execute flag replaced with --dry-run. Dry runs will confirm with PTOSC whether or not the command is ready before the original table is modified.

LOM tries to be flexible: not changing queries unnecessarily and supporting common ‘raw’ queries as well. So dropping a table won’t go through PTOSC, or if migrations rely on hand-written SQL then they should work without human intervention. For example a raw query like \DB::statement("ALTER TABLE my_table CHANGE fruit fruit ENUM('apple', 'orange')"); will be translated to a PTOSC command, while \DB::statement("DROP TABLE my_table CASCADE"); will remain unchanged.

Fine-grained control of which online tool–if any–is used can be found within the configuration file config/online-migrator.php, environment variables like ONLINE_MIGRATOR, and traits on the migration scripts themselves. For more see the documentation on usage. Also of note, the output of “php artisan migrate” will be more verbose in order to aid resolving problems with migration runs.

UPDATE 2019-02-05: Forgot to mention the convenience option doctrine-enum-mapping was included to make changing tables with DB enumerations easier. By setting its value to ‘string’ migrations can use Eloquent code to change enum-equipped tables, though yet not for changing the enum columns themselves.

If this has been helpful please consider commenting here or opening an issue or pull request on the project’s Github.

NOTE: All opinions and thoughts expressed here are my own and do not reflect those of my employer.

Acer Predator Laptops Not For Ctrl Swappers Or Cap-Alt-ers

If you’re someone who likes to move the Ctrl key to your home row using Caps Lock, or you like custom Caps+Alt macros, then Acer’s gaming laptops may not for you. Unless of course you don’t need Ctrl+Alt+A or Caps+Alt+A, because that that physical caps-with-alt-a combo is the one and only that won’t work:

Most users may think this is a very niche complaint. But password managers like KeePass default to the Ctrl+Alt+A combo for auto-typing, a feature which may be the only alternative when paste blocking and in-browser add-on’s won’t do. Ctrl+Alt combos are also useful for streaming and power users who want global hotkeys that break out of the scope of programs on Microsoft Windows. Gaming too often requires pressing many keys at once, so much so that the term N-key Rollover was coined. And since the Predator series of laptops targets gaming, it is strange to see such a oddly narrow gap in models that cost $2,000+ USD.

Caps-as-ctrl is also a fairly well known arrangement, so much so there is even a highly ranked article disputing its benefits. It’s an arrangement I’ve enjoyed on-and-off for years, and until now had never seen a keyboard which did not allow the physical caps-alt-a combo.

If you’re a consumer consider yourself warned. If you’re a manufacturer then please better educate your level-1 support, and thoroughly test features important to your target demographic. Because top-shelf consumers may go elsewhere when faced with beta testing your hardware.

Uneven Game Difficulty Hinders Accessibility

Getting stuck in a game can be an opportunity to rise to the challenge or be so annoying it feels like a waste of time to keep trying again and again. Everyone has their own threshold and unique mixture of abilities, which makes well-balanced difficulty options so critical to a product’s accessibility.

Boss fights in the original release of Deus Ex: Human Revolution were unpopular because they broke the difficulty curve so badly. For a game series with a reputation for player choice the game launched with only one way to defeat each boss, and with lethal force being the only option. Thankfully later updates provided more options, making it more consistent with the rest of the experience.

Too often the difficulty adjustments in games fall into similar traps when they focus solely on easy tweaks, like making enemies easier to eliminate or the player more resilient. Other game mechanics or mini-games such as puzzles are left largely unchanged which breaks the flow for players more interested in exploration, or less clever than their compatriots. And while I respect designers’ desire to craft an experience and set expectations, it’s hard to justify inconsistent difficulty levels as gaming audiences grow to include people with less free time and varying abilities.

Watch Dogs is another example of a game which feels like it’s only difficult adjustment is rudimentary nerfing, buffing, and timer tweaks. While I appreciated the mission variety and many of the challenges, aspects like tightly timed chases, enemies with supernatural bursts of speed, and nearly impossible to escape police and ‘fixers’ were often infuriating. Even the age-old tradition of cheat codes were unavailable here. Some game guides resorted to suggesting workarounds like shooting out a window then hiding for notoriety boasts, or swimming out into open water to escape.

Considering the resource constraints of smaller development shops it is easier to accept uneven difficulty in their releases. However, larger studios have no excuse; except perhaps ignorance, greed, or laziness. In this age of day-one patches there is little reason not to at least update a product found to have uneven gameplay, even if only to patch in some cheats. Consumers should not have to resort to 3rd-party hacks developed by the community to fully enjoy their purchase.

What do you think? What has your experience been with games and their difficulty levels?

Single Player Commandments

As someone who enjoys older games a story campaign is often critical since multiplayer communities have typically already moved on. Yet it’s hard to overlook some advances in game usability when going back to the past. Now technically these commandments are meant to address campaigns, whether solo or co-op. Still, whether making a new game or remastering an old one, designers please consider these guidelines to avoid frustrating your players.

I. Thou shalt not require Internet
An Internet connection requirement for a campaign is the bane of traveling and rural players who may not have the access or bandwidth to enjoy your game. Neglecting local-network and couch co-op is similarly frustrating for these gamers.

II. Thou shalt always allow pause and skip
Being able to pause at all times is important to gamers with families and responsibilities that may interrupt play. Similarly being able to skip cut-scenes or non-core gameplay helps avoid frustration. Otherwise one may feel forced to miss critical scenes or re-experience the same thing over and over. Ideally players can even rewind, replay, and fast-forward non-interactive cut-scenes.

III. Thou shalt auto-save frequently
Automatically saving at milestones is one helpful feature of many modern games. More frequently is better, though only when there is more than one auto-save slot. If nothing else then auto-saving when a player exits—even during a cut-scene—is better than burdening players with the fear-of-missing-out.

IV. Thou shalt quick save and load anywhere
Providing easy and customizable access to a quick save and quick load is even better than auto-save. It functions as a kind of permanent pause and resume helping players feel they are in control of the experience. This too should include saving and reloading in the midst of a cut-scene.

V. Thou shalt allow changing difficulty at any time
Easier difficulty modes are essential for busy and novice gamers, yet without the ability to change the difficulty players can be trapped hours into a path ultimately too boring or frustrating for them. If resources allow then the option to automatically nerf or buff the difficulty is best, so long as the player can override this in case they find dynamic difficulty doesn’t suit them.

VI. Thou shalt provide navigation hints
Whether arrows, way-points, hot-spot indicators, or objective menus there must always be guides for players who so desire. Of course these may be reduced or completely removed when players want a greater challenge.

Please take care not to rationalize transgressing these commandments as ‘essential’ features of your game. As a designer the intended experience should be a goal—not an unmovable absolute. Also keep in mind that neglecting accessibility and usability reduces the audience capable of appreciating your games.

Did I miss a commandment? Is there one that doesn’t belong? Comments are always appreciated.

How To Export Contacts From OwnCloud Database

For those who need or prefer directly dumping all their contacts from an OwnCloud database, below are commands to do it from a PostgreSQL installation.

Remember to put your OwnCloud user-name in place of the “…” in the query.

psql owncloud --tuples-only --no-align -c \
"SELECT encode(carddata, 'escape') FROM oc_cards WHERE addressbookid = (SELECT id FROM oc_addressbooks WHERE principaluri = 'principals/users/...')" \
> oc_cards.vcf

If you happen to have accented, or other non-ASCII, characters then you may need to convert from octal (“\nnn”) to UTF before trying to import elsewhere.

Why Can’t I Change Gamepad Controls?

A disturbing trend among some video games is gamepad support whose controls cannot be customized. While the standardization of gamepad support on PC’s has increased the number of games supporting them, for some games there is only a single configuration. For those of us who are differently abled, or simply prefer to use a familiar layout, control customization seems to be taking a step backwards.

This is surprising since personalizing controls has long been a feature of PC gaming for decades, even among small budget titles. Consoles such as the PS4 and Xbox One now offer the ability to remap keys for all games. This is a modest accessibility improvement, though it appears to come at the cost of less in-game remapping. So players who prefer to alternate between different games the situation often involves re-configuring one’s brain each time. One would hope the competition among PCs and various console platforms would drive progress towards more accessible controls.

It’s true that enterprising users and 3rd-party developers provide alternative means of remapping controls. Sadly, many of these fail to remap for multiple games at once. Steam is the most accessible and broadly available which does do per-game mapping. Still, being outside the game requires users know their games default controls and do the old-to-new mapping using only the abstract button names. It’s also a relatively unknown feature.

Despite the variety of games and genres there is a lot of similarity in game control: move forward-back-left-right, jump, crouch, action/shoot, sprint, etc. On PC the keyboard controls have defaulted to the WADS keys for forward-left-right-back, so why not have a means to change the defaults for all games at once? Certainly some games will have unique controls which cannot be standardized. In those cases in-game customization may be the best solution. Still, it would save gamers time and frustration if they could begin with familiar fundamentals when starting or switching among experiences.

If PCs and consoles increasingly become home-theater machines and gamers play a larger variety of games they too may be asking why controls are so difficult to personalize. Hopefully developers will take notice.

Thinking Of Password Strength Like Slot Machines

Instead of communicating password strength—specifically unpredictability—as ‘bits,’ let’s consider using slot machines as a metaphor. Bits of entropy are the traditional way of describing the randomness and variety, though it can be intimidating or too abstract for some users. It can also overlook that each ‘character’ in the equation is actually an entire word, since those are easier for users to remember. Slots are relatively simple machines which randomly rotate among variations of options and line up together like letters or words in a passphrase. So this metaphor may help users better understand the strength of their passwords.

Let’s imagine that slot players are thieves looking to steal the money in your bank account. Slot positions or wheels can be thought of as each character or word in your password or phrase. And the varieties of slot fruits represent the possible characters or words in your secret. Now if a player happens to line up each of them they’ll strike jackpot and get access to all your money. Preventing unwanted jackpots like that requires choosing passwords or phrases that are unpredictable. So we want a lot of different fruit (characters/words), as many slot positions as practical (as long as possible), and no pattern among the slot choices (no relationship between them). More is stronger.

It’s important to keep in mind that if we choose a password as a collection of words (not random characters) then our ‘fruit’ or dictionary cannot be counted as each character. In that case it must be counted as each whole word. Attackers know that people often use words because of frequent passwords leaks. So any pattern we choose should be considered public knowledge.

Now, to gauge how strong a password (jackpot) is we can multiply the number of possible words/characters (fruit) together repeatedly, one for each slot position. Consider that three words with two slots is 3 x 3 or 9 possible values. Put another way we take it to the exponent: 3 ^ 2 = 9. Guessing randomly the attacker/player would probably hit the jackpot in half as much, on average. So that password would typically require 9 / 2 or 4.5 guesses to get right. That’s really weak.

Expressing that as attempts gets out of hand quickly since at 1000 words/characters with 3 slots we’ve got 1,000 ^ 3 or 1,000,000,000 which is one billion. A dictionary or fruit basket of 10K with 5 slots would be 100,000,000,000,000,000,000 or one hundred quintillion, on average guessed in fifty quintillion attempts. If these numbers are too unwieldy we could use the ‘bits’ method to make these numbers easier to read and reason about. Then our 10,000-5-slot password would be log base 2 of 100 quintillion or about 66.4 bits. Still it would be important to remember to take care when filling in the ‘fruit’ or dictionary number to avoid counting characters when we’re actually using words.

P.S.

Each of the password requirements (many kinds of ‘fruit’, multiple slot positions, unrelated ‘fruit’ choices) is important, and that makes strong password/phrase choices hard to remember. Typically we also have many accounts with a variety of website and services. Ideally each account should use a different, unrelated password than all the others. For this reason a password manager makes life more practical. Properly designed they can encrypt all the random passwords or phrases with one, strong-yet-memorable password. If you aren’t using one please try making it a part of your daily routine for a month. It could save you a lot of time and headache by avoiding the loss of your accounts or funds.

“User-Agent” Headers Holding Back The Web

Every time you visit a website the name and version of your browser is sent to the service. In fact with every requested image, video, and style sheet the same data is sent again and again. This not only wastes bandwidth, it also subtly encourages web makers to rely upon it as a shortcut to make services work consistently across platforms. Later browsers then include more tokens in their “User-Agent” header to maintain compatibility with these fragile services. Over time the header becomes larger and the web more brittle. For example, Internet Explorer 11 identifies itself as “Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko”. Can you tell which part communicates that it is Microsoft’s Internet Explorer?

Of course it’s impractical for every web site/service to test every possible combination of browsers and platforms. So those of us developing sites and services only test the most popular browsers at the moment. Over time this leads to a web which caters to a mix of the most popular browser of the past and present, depending upon the time any given service was last made. As more and more devices leverage HTTP for the Internet-of-things this problem may grow more complex. Web standards and feature detection can help.

With well defined standards and run-time detection of features it’s possible to avoid the trap of ‘sniffing’ the browser from it’s UA headers. And while cutting edge features and services may benefit in the short-term from taking the shortcut of browser detection, they can also leverage vendor-specific prefixes of features in flux. Once standardized the prefixes can be replaced with official and non-prefixed names.

My experience detecting significantly different platforms such as mobile or internet-of-things (IOT) devices do still have some valid uses for the UA header. But ultimately they may be better served by a new, simpler header or more platform-independent designs. Until then Mozilla’s recommendations are a reasonable place to start.

In recent years even the once-dominant Microsoft notes the weaknesses and problems with UA headers. Sadly, my experiments sending an empty or minimal UA header have found too many sites broken to recommend the approach to non-technical users.

How about you? What do you think of UA headers?

Is Ad Blocking A Form Of Looting?

One description of the increase in ad blocking is that it’s a kind of boycott. While that may be the view of blockers the content producers may see it differently. For them advertising pays for their effort to create the content. So when people consume their content without any payment (in the form of attention) then their incentive to produce suffers.

This ‘boycott’ of advertising—while still taking the ad-supported services—appears to have much more in common with looting. Looting often occurs when large numbers of people feel deprived or exploited. Like a kind of vigilante justice one could argue the careless and invasive advertisers have pushed users to this extreme.

Of course a boycott sounds a lot more noble. But if I’m not mistaken that would involve avoiding the entire business, not using their services while dodging payment. Picketing is another feature that can accompany boycotts. Those bothered by ads do have options to express their discontent on social media, forums, and comments; though, admitted not always directly on the site of the business itself.

Alternatives to ad-funding do hold promise: micro payments, donations, and subscriptions. Each has some friction for users to get involved. Perhaps when doing so is easier than installing an ad-blocker things will turn around.

So what do you think? Is ad blocking more like a boycott or looting? Something else entirely?