Base data creation:

CREATE
  (l:List {id: 42, name: "Stuff"}),
  (i1:Item {id: "a", description: "Fry"}),
  (i2:Item {id: "b", description: "Bender"}),
  (i3:Item {id: "c", description: "Leela"}),
  (i1)-[:IN_LIST {position: 0}]->(l),
  (i2)-[:IN_LIST {position: 1}]->(l),
  (i3)-[:IN_LIST {position: 2}]->(l);

Cypher statement to move an item by its id property and 'swap' it with another item with id c. Here, 'swap' doesn't mean the literal meaning of swap, which would be to exchange the indices of the appropriate items. Moving an item to an item that is currently before it means that it gets moved before that item, while moving an item to an item after it means that it gets moved after that item. The net effect is a full range of movement possibilities.

MATCH
  (l:List {id: 42}),
  (i1:Item {id: "a"})-[r1:IN_LIST]->(l),
  (i2:Item {id: "c"})-[r2:IN_LIST]->(l)
WITH
    r1.position AS oldPosition,
    r2.position AS newPosition,
    r1 AS r1,
    CASE
      WHEN r2.position < r1.position THEN 1
      WHEN r2.position > r1.position THEN -1
      ELSE 0
    END AS signum
MATCH (i:Item)-[r3:IN_LIST]->(:List {name: "Stuff"})
WHERE
  (newPosition < oldPosition
 AND r3.position >= newPosition AND r3.position < oldPosition)
 OR (newPosition > oldPosition
AND r3.position > oldPosition AND r3.position <= newPosition)
SET r2.position = r2.position + signum, r1.position = newPosition;
Posted 2020-10-14

A series of meaningless tokens.

urn:ietf:wg:oauth:2.0:oob

There's also one called urn:ietf:wg:oauth:2.0:oob:auto.

Posted 2020-10-07

Need a quick HTTP server? Here is one.

import http
import http.server

class MyHandlerClass(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        print("Inside GET callback")
        self.send_response(http.HTTPStatus.OK)
        self.send_header('Content-Type', 'text/plain')
        self.end_headers()
        self.wfile.write(b'Hello, world!')


s = http.server.HTTPServer(
    ('localhost', 49152), MyHandlerClass
)
s.serve_forever()
Posted 2020-10-07

The lowest UUIDv4 is as follows:

00000000-0000-4000-8000-000000000000

Good to know, for many purposes.

Posted 2020-09-24

This is very old-school webdev, but I recently had reason to create a styled placeholder. Or specifically, a placeholder where part of the placeholder text has a different style. This doesn't seem possible to do reasonably without vendor-prefixed web APIs. Here's how you can do it.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Conforming HTML5 Template</title>
<link rel="shortcut icon" href="favicon.ico" type="image/x-icon">
<style>
#sp-container {
    position: relative;
}

#sp-placeholder {
    position: absolute;

   /* non-essential; mimic default placeholder */
    font-family: sans-serif;
    font-size: 14px;
    color: #777777;

    /* vertically centre */
    top: 50%;
    transform: translate(0%, -50%);
}
</style>
<script>
const PLACEHOLDER_ID = 'sp-placeholder';
const INPUT_ID = 'sp-input';

function onReady() {
    const span = document.getElementById(PLACEHOLDER_ID);
    const input = document.getElementById(INPUT_ID);

    const maybeShow = () => {
        if (input.value === "") {
            span.style.display = 'inline-block';
        }
    };

    const hide = () => {
        span.style.display = 'none';
    };

    // If the value was pre-filled by some query param, don't show the hint.
    const maybeHide = () => {
        if (input.value !== "") {
            hide();
        }
    };

    // To hide on focus:
    //input.addEventListener('focus', hide)
    input.addEventListener('input', hide)
    input.addEventListener('blur', maybeShow);

    maybeHide();
}

document.addEventListener('DOMContentLoaded', onReady);
</script>
</head>
<body>
<form id="sp-container">
<label>
<span id="sp-placeholder">Enter text <strong>here</strong></span>
<input type="text" id="sp-input"/>
</label>
<input type="submit"/>
</form>
</body>
</html>

Several things to note here. This uses the relative-inside-absolute trick to take the placeholder span out of the main flow. Note that if the form is submitted, and the user then presses the back button, the form values will be recalled by default on page load. In this case, you should hide the placeholder span. I'm not totally sure that this solution actually handles that properly, although it attempts to.

The vertical centring can be handled either through the top/translate trick shown above, or it can be handled through tricky margins and calc(). I didn't have much luck with flexbox in this case, although it may also be a possibility.

Posted 2020-07-23

A few arbitrary tips on Omeka-S theming.

Retain existing classes on analogous elements as far as possible, as they are sometimes referred to by internal Omeka CSS classes. Obviously you can imagine edge cases where this isn't desirable. But it's a good first rule to follow. Obviously if you omit some design-element entirely you must also omit the class associated with it.

In a similar vein, be wary of doing less than the requirements of a theme. There's no real verification or way of testing that a theme handles all paths sensibly and in a manner consistent with the expectations of the user.

For example, there are certain site settings your theme should support: "embed media on item page" is a site specific setting that you must handle within your theme, but there is no validation that your theme does in fact handle this.

Do not ever use <p></p> elements in your module markup. Use custom divs or spans with your own classes. Why? The Html blocklayout generates <p> tags, so any attempt to style page text will often inadvertently style your more-irregular p tags.

Config form names should be in snake_case.

Debugging templates: a very rudimentary debugging method is to fill your partials with introductory HTML comments naming their source file.

Eg:

<!-- BEGIN view/omeka/site/item/show.phtml -->
<p>This is content this is content this is content</p>
<!-- END view/omeka/site/item/show.phtml -->

One way to modularize your CSS is to break out the per-page CSS into individual CSS files, then include them in layout.phtml

$this->headLink()->appendStylesheet($this->assetUrl('css/style.css'));
$this->headLink()->appendStylesheet($this->assetUrl('css/item-show.css'));

Here, item-show.css contains rules associated to the item detail page.

Don't try to remove these from your layout.phtml. These are internal Omeka files that in turn may cause the loading of more than one JS file.

$this->headScript()->prependFile($this->assetUrl('js/global.js', 'Omeka'));
$this->headScript()->prependFile($this->assetUrl('vendor/jquery/jquery.min.js', 'Omeka'));
Posted 2020-07-10

Patreon seem to find it acceptable to do a whole bunch of security-through-obscurity measures on their site that verge on DRM. This is a symptom of the times, really. They don't provide an API for users, although they do provide an API for creators. Let's try to scrape MP3s.

<a data-tag="post-file-download"
   href="https://c10.patreonusercontent.com/LONGSTRING" class="sc-fzpans kmqqXw">Myfile.mp3</a>

This is what the link for downloads looks like. We are after the URL in the href attribute, but we won't be able to get it without providing our cookie. A query string parameter token-hash and token-time are included in the URL. This means that it's actually possible to download these files without sending any cookie, as the authentication details are already encoded in the URL.

So that means that the problem reduces to the following:

  • Expanding the infinite scroll of the site
  • Find all links and correlate them with some metadata

Expand the infinite scroll

Start out at the URL that has a tags filter that matches what you want.

https://www.patreon.com/somecreator/posts?filters%5Btag%5D=Patron%20Only%20Audio

We look for the 'Load more' button and expand it. Make sure you actually physically scroll down the page until the first instance of this button appears. el.children.length allows us to choose the deepest element in the hierarchy before going upwards to its parent. This prevents us from accidentally picking an outer container element which would also transitively contain the target text, Load more.

function findLoadMore() {
     const buttonMatches = Array.from(document.querySelectorAll('div')).filter(el => el.textContent === 'Load more' && el.children.length === 0);
     if (buttonMatches.length === 0) {
         return undefined;
     }

     if (buttonMatches.length > 1) {
         throw new Error("ambiguous");
     }

     const theMatch = buttonMatches[0];
     const theButton = theMatch.parentNode;

     if (theButton.tagName !== 'BUTTON') {
         throw new Error("unexpected");
     }
     return theButton;
}

function expandScroll() {
    const button = findLoadMore();
    if (button === undefined) {
        console.log("terminating expansion");
    } else {
        console.log("click");
        button.click();
        window.setTimeout(expandScroll, 30000);
    }
}

The site is so ungodly slow that this can hang Chrome quite easily. I suggest either having an unrealistic amount of patience, or pausing script execution in the "Sources" tab in the inspector after a while.

Scrape the URLs

Scraping the URLs is a bit more interesting. We need two pieces of information, the first is the post publication date:

<a data-tag="post-published-at" href="/posts/some-post-12345" class="sc-fzpans bwkMGo">May 16 at 9:28am</a>

There may be more information in the show title but I don't think that we can guarantee uniqueness purely from use of that information. The second piece of information is the download URL.

From some experimentation in the inspector, I can tell that the common ancestor of these elements is the following: <div data-tag="post-card">...</div>

This at least is a bit semantic.

Let's scrape:

var postNodes = document.querySelectorAll('[data-tag="post-card"]');
var data = Array.from(postNodes).reduce((result, el) => {
    const publishedAt = el.querySelector('[data-tag="post-published-at"]');
    const postFileDownload = el.querySelector('[data-tag="post-file-download"]');

    if (postFileDownload !== null)  {
        result.push({
            publishedAt: publishedAt.textContent,
            href: postFileDownload.getAttribute('href')
        });
    }
    return result;
}, []);

We have to handle the case where we don't find any data in case of mis-tagged items.

Now you can console.log(JSON.stringify(data)) and paste it into a text editor, you can save it as exported.json. Using that JSON data you can use this Python script to download the files into timestamped files:

import json
import requests
import unicodedata
import re

with open('exported.json', 'r') as f:
    data = json.load(f)

def slugify(value):
    value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode('ascii')
    value = re.sub('[^:\w\s-]', '', value).strip().lower()
    return re.sub('[-\s]+', '-', value)


for rec in data:
    timestamp = rec['publishedAt']
    href = rec['href']
    filename = "{}.mp3".format(slugify(timestamp))
    print(filename)
    r = requests.get(href, stream=True, timeout=30)

    with open(filename, 'wb') as fd:
        for chunk in r.iter_content(chunk_size=128):
            fd.write(chunk)
Posted 2020-05-17

I wanted to make an example getting started post as a reaction to an official vertical list example which I find to be overly complex.

What API surface does it provide? It provides a bunch of things:

DragDropContext, this is a top-level container element. Droppable, which must envelop all draggables. Draggable, which appropriately enough must envelop an individual draggable item.

Each item expects its children prop to be a function, which it calls to get further elements to render. That enables it to pass the context information down the component tree. In some ways this is a good example of the issue I described earlier in the post FP & the Context Problem.

Luckily this library has a good set of types available in the DefinitelyTyped repository. You can find out most of the API from reading the TypeScript definitions. So a well-typed example is below.

import React, {useState} from 'react';
import {
    DragDropContext, Droppable, Draggable,
    DropResult, DroppableProvided, DroppableStateSnapshot, DraggableRubric, 
    DraggableStateSnapshot, DraggableProvided, ResponderProvided
} from 'react-beautiful-dnd';

interface AppProps {
};

function makeDraggableChildren(value: string) {
    return (provided: DraggableProvided,
            snapshot: DraggableStateSnapshot,
            rubric: DraggableRubric) => 
                <li ref={provided.innerRef} 
                     {...provided.dragHandleProps}
                     {...provided.draggableProps}>{value}</li>;
}

function makeDroppableChildren(items: string[]) {
    return (provided: DroppableProvided, 
            snapshot: DroppableStateSnapshot) =>
                <ul ref={provided.innerRef}>
                  {items.map((value, i) => (
                      <Draggable draggableId={i.toString()}
                                 index={i}
                                 key={i}>
                        {makeDraggableChildren(value)}
                      </Draggable>
                  ))}
                  {provided.placeholder}
                </ul>;
}

export function DndDemo(props: AppProps) {
    const [items, setItems] = useState(['fry', 'bender', 'leela']);

    const onDragEnd = (result: DropResult, provided: ResponderProvided) => {
        if (!result.destination)  return;    // invalid drop

        const startIndex = result.source.index;
        const endIndex = result.destination.index;

        // Destructure because we know we always have 1 item.
        const [removed] = items.splice(startIndex, 1);
        items.splice(endIndex, 0, removed);
        setItems(items);

        console.log("start index is %o, end index is %o", startIndex, endIndex);
    };

    return (
        <div>
          <h1>Hello</h1>

          <DragDropContext onDragEnd={onDragEnd}>
            <Droppable droppableId="main">
              {makeDroppableChildren(items)}
            </Droppable>
          </DragDropContext>
        </div>
    );
}

While it's still very ugly, I feel that overall this is a relatively good deal when it comes to avoiding dealing with the HTML5 drag and drop API directly. The abstraction is extremely leaky-to-nonexistent here, as you can see. You're plugging a bunch of mandatory stuff into your HTML and while you can ignore the detailed-contents of the stuff, you can't ignore the presence of the stuff itself. Every innerRef, *props value here is the cruft from the mechanism showing through. I must give this library its due, though: the error messages from this library are extremely good, and the documentation is also decent. Once you commit to introducing this piece of large bit of ugly boilerplate, you do get a good amount for taking on this burden, so I think it's a good bet overall.

I feel like there's probably a much more idiomatic way to do the currying approach above, but I'm not experienced enough with React yet to know what it is.

Posted 2020-05-12

I don't know that there's a good name for this technique, but it's basically what's required when you need to have local database credentials that differ from other developers in a team, and also can't be committed to version control. In React and Vue this feature is called .env.local.

When you need this in Spring, you need to know the following:

  • You can have multiple profiles.
  • But you'll need to end up gitignoring a file pattern at some point anyway.
  • Your 'mainstream' config file is likely stored under your project tree at src/main/resources/application.properties.
  • The default search path for Spring is:
    • file:./config/
    • file:./
    • classpath:/config/
    • classpath:/
  • file:. means the real root of your project, i.e. where your .git directory is, pom.xml, etc.
  • classpath:/ means src/main/resources, broadly speaking.

From this we can reason that the path of least resistance is to add a gitignore rule /config/ (the leading slash is important), and store your custom configuration in config/application.properties in the project root. Of course the more sophisticated way would be to use actual profiles, but this has the benefit of not needing any IDE configuration.

Note that this tends to work on a MERGE basis, rather than a simple basis where you use one config OR another config.

For instance, imagine that the standard application.properties as used by the team contains these lines:

logging.level.org.springframework.jdbc.core.JdbcTemplate=DEBUG
logging.level.org.springframework.jdbc.core.StatementCreatorUtils=TRACE

Now imagine you have a similar set of lines in your own config/application.properties file:

logging.level.org.springframework.jdbc.core.JdbcTemplate=DEBUG
logging.level.org.springframework.jdbc.core.StatementCreatorUtils=TRACE

The root logger for the application is at INFO level. You won't be able to silence these logs just by commenting out the lines in your own file. Because that will mean that the previous configuration remains, as it wasn't overridden by any more specific rule from your personal configuration. You need to explicitly override the log level for these classes:

logging.level.org.springframework.jdbc.core.JdbcTemplate=INFO
logging.level.org.springframework.jdbc.core.StatementCreatorUtils=INFO

To set the root logging level, use:

logging.level.root=DEBUG

Do not attempt to use the "short cut" logging.level., this will not work.

Posted 2019-11-26

I know next-to-nothing about any of these, except for Typescript which I know a bit about. The goal is to produce the most minimal example possible of a JS app with state, i.e. a counter, the classic demo.

Set up the project with npx create-react-app myapp --typescript.

First off: You want both pieces, redux and react-redux. You will use imports from both of these.

To get data into and out of your components, you'll need a store. The store is constructed with the createStore function from redux.

You always need a reducer to construct a store. A reducer is a function from a state and an action to the next state.

This should be the contents of index.tsx:

interface MyState {
    counter: number;
}

const INCREMENT = 'INCREMENT';

interface IncrementAction {
    type: typeof INCREMENT
}

type MyActionTypes = IncrementAction;


function myReducer(state: MyState | undefined, action: MyActionTypes): MyState {
    console.log("Reducer being called.");
    if (state === undefined) {
        return { counter: 0 };
    }

    switch (action.type) {
        case INCREMENT:
            return Object.assign({}, state, { counter: state.counter + 1 });
        default:
            return state;
    }
}

const store = createStore(myReducer);

ReactDOM.render(
    <Provider store={store}>
        <App />
    </Provider>,
    document.getElementById('root')
);

Points to note here:

  • MyActionTypes is a sum type with only one member.
  • Each action has a mandatory property, type. The use of typeof in the definition of IncrementAction ensures that type receives a string literal type. This in turn enables the switch below to be type safe.
  • The structure of the switch will make sure that action is inferred to the correct type within its case branch. That is, even though MyActionTypes may be Foo | Bar, within its matching case branch TS knows that action is FooAction or BarAction.
  • Provider is a wrapper component that will wire up the store to all components that are beneath it in the component tree.
  • The type of the first argument must be MyState | undefined NOT simply MyState. You react to the undefined state by configuring the initial state. You're going to get confusing type errors when you try to call createStore if you type this wrongly.

This should be the contents of App.tsx. Note that you also need the type definitions from above, I recommend extracting them to a file.

function mapStateToProps(state: MyState) {
    return {
        counter: state.counter
    };
}

function incrementActionCreator(): IncrementAction {
    return {
        type: INCREMENT
    };
}

const mapDispatchToProps = {
    increment: incrementActionCreator
};

interface AppProps {
    counter: number;
    increment: () => void;
}


class App2 extends React.Component<AppProps> {
    render() {
        const { counter, increment } = this.props;

        return (
            <div className="App">
                <header className="App-header">
                    <img src={logo} className="App-logo" alt="logo" />

                    <p>Counter value: {counter}</p>

                    <button onClick={increment}>Increment</button>

                    <a
                        className="App-link"
                        href="https://reactjs.org"
                        target="_blank"
                        rel="noopener noreferrer"
                    >
                        Learn React
                </a>
                </header>
            </div>
        );
    }
}


export default connect(
    mapStateToProps, mapDispatchToProps
)(App2);

Things to note here:

  • The connect function lives in react-redux.
  • App2 needs to be converted to the class-based component syntax, it won't work with the functional component syntax, as far as I can see.
  • An action creator is a function returning an action, which in this case just means that it returns an object that has a type property.
  • The argument to mapStateToProps tells react-redux that the prop counter, accessible within App2 as this.props.counter, should reflect the value of the property counter in the store. (The fact that they have the same name is incidental.)
  • The object mapDispatchToProps just says: within App2, the function-valued prop increment will call the action creator incrementActionCreator and dispatch the resulting action... which will end up calling the reducer.
  • To get these props typed, we have to explicitly declare them by making our component a class that extends React.Component, and declare an interface AppProps. I'm not sure that the type of increment is correct, but it's marginally better than the any type, I would suppose.
  • Arguably the declaration of AppProps should be unnecessarily as it should be able to be inferred; here, React is in a rather similar position to the Vuex helper methods such as mapState. Neither provide automatic typing of mapped store props, I was rather hoping that React might have been better here -- but see this question.
  • Note that we have to destructure the mapped stuff from this.props. This is a little bit dangerous because if you used the same name for the mapped prop as the action creator (which you've probably imported into your scope from Some other module), you're shadowing that with the mapped-prop version, meaning that if you FORGET to destructure the props and just end up calling the parent-scope version then you'll get no behaviour and no errors.
Posted 2019-10-29
SSH key setup
Posted 2019-09-11
Stretch to Buster
Posted 2019-08-05
Subprocess Pipe Comparison
Posted 2019-07-02
The X3 Wiki Archive
Posted 2019-06-16
Fabric 2 cheat sheet
Posted 2019-03-05
Using comboboxes in Qt5
Posted 2019-02-27
System Puppet, CentOS 7 Client
Posted 2019-02-25
X3 savegames
Posted 2019-02-02
Shadow Tween technique in Vue
Posted 2019-01-06
Width list transition in Vue
Posted 2018-12-18
Emoji Representations
Posted 2018-09-14
Thoughts on Cheesesteak & More
Posted 2018-08-29
Vue + GraphQL + PostgreSQL
Posted 2018-07-20
Neo4j Cypher query to NetworkX
Posted 2018-05-09
FP & the 'Context Problem'
Posted 2018-02-27
Cloake Vegetable Biryani
Posted 2018-02-25
FFXII Builds
Posted 2018-02-02
Custom deployments solution
Posted 2017-12-09
SCons and Google Mock
Posted 2017-11-30
Sunday Lamb Aloo
Posted 2017-11-19
centos 6 debian lxc host
Posted 2017-11-03
Srichacha Noodle Soup
Posted 2017-10-17
Kaeng Kari
Posted 2017-10-13
Ayam Bakar (Sri Owen)
Posted 2017-10-12
Pangek Ikan (Sri Owen)
Posted 2017-10-12
Chicken Tikka Balti Masala
Posted 2017-10-06
Clojure Log Configuration
Posted 2017-09-28
Clojure Idioms: strict-get
Posted 2017-09-28
About
Posted 2017-09-18
Philly Cheesesteak
Posted 2017-09-14
Welcome
Posted 2017-09-13
Srichacha Kaeng Pa
Posted 2017-08-31
Malaidar Aloo
Posted 2017-08-10
BBQ Balti Chicken
Posted 2017-07-19
Sabzi Korma
Posted 2017-07-18
Vegetable Tikka Masala
Posted 2017-07-02
Soto Ayam
Posted 2017-06-08
Bombay Aloo w/Bunjarra
Posted 2017-06-03
Chicken Dopiaza
Posted 2017-06-01
LJ Bunjarra
Posted 2017-05-31
Glasgow Lamb Shoulder Tikka
Posted 2017-05-24
Tofu Char Kway Teow
Posted 2017-05-12
King Prawn Balti
Posted 2017-04-24
Ad-hoc Quorn Rogan Josh
Posted 2017-04-15
Glasgow Vindaloo
Posted 2017-03-28
Rempeyek
Posted 2017-03-26
Toombs Saag Balti
Posted 2017-02-25
Glasgow Bombay Rogan Josh
Posted 2017-02-21
Glasgow Chicken Balti
Posted 2017-02-16
Quorn Balti & Cloake Naan
Posted 2017-02-03
Two Spice Marinades
Posted 2017-01-18

This blog is powered by coffee and ikiwiki.