River4 http://river4.smallpict.com/ Tue, 01 Dec 2015 15:50:52 GMT Tue, 01 Dec 2015 15:50:53 GMT en-us Fargo v1.71 http://cyber.law.harvard.edu/rss/rss.html dave.winer@gmail.com davewiner dave.winer.12 Using River4 to flow news to Pinboard http://river4.smallpict.com/2015/11/08/usingRiver4ToFlowNewsItemsToPinboard.html <p><a href="https://github.com/scripting/river4/wiki/How-callbacks-work-in-River4">In June</a> I added the ability to "River4" to write JavaScripts that run when a new item is added to a river. The run from a folder, so they're easy to edit. You can have as many as you like. They can do anything a Node.js app can do, they even have persistent storage. And they can modify the data that River4 stores. We gained a lot of experience from these kinds of callbacks working on earlier River software in the Frontier environment.<img style="float: right; margin-left: 25px; margin-bottom: 15px;" src="http://scripting.com/2015/10/05/goodbyeBlueMonday.png"></p> <p>Yesterday, <a href="https://groups.google.com/forum/?fromgroups#!topic/river4/gTjLGr0FNqk">Nicolas Meier released</a> a River4 <a href="https://github.com/nicolasm/river4AddToPinboard">callback</a> that sends all items in a river to a <a href="https://pinboard.in/api/">Pinboard</a> account. This makes it possible to use the <a href="http://cyber.law.harvard.edu/rss/rss.html">RSS</a> feeds to populate a searchable database of news items. A good demo of the callback facility and feed technology.</p> <p>If you have questions, ask on the <a href="https://groups.google.com/forum/?fromgroups#!forum/river4">River4 mail list</a>. </p> <p>And thanks to Nicolas for this contribution! <img title=':blush:' alt=':blush:' class='emoji' src='http://fargo.io/code/emojify/images/emoji/blush.png' align='absmiddle' /></p> Sun, 08 Nov 2015 18:52:40 GMT http://river4.smallpict.com/2015/11/08/usingRiver4ToFlowNewsItemsToPinboard.html Editing subscription lists http://river4.smallpict.com/2015/08/04/editingSubscriptionLists.html <h4>The lists folder</h4> <ol> <li><p>A subscription list is an OPML file in the <i>lists</i> folder of your "River4" data folder. </p></li> <li><p>Each list corresponds to a river. The feeds in the list are the feeds we read to find new items for the corresponding river. </p></li> <li><p>Suppose you wanted a river of news about movies. You'd create an OPML file in the lists folder called movies.opml. River4 would read the feeds in that list periodically, and put the new items in the <a href="http://riverjs.org/">riverjs</a> file in the <i>rivers</i> folder. The river file is called movies.js. </p></li> <li><p>To edit the list you can use a text editor, to hand-edit the OPML, or you can use an outliner whose native format is OPML. </p></li> </ol> <h4>Using Fargo to edit OPML lists</h4> <p>My outliner, "Fargo" is perfectly suited for editing OPML subscription lists. </p> <ol> <li><p>First create a new outline, using the File/New command in Fargo.</p></li> <li><p>Create a new headline by pressing Return. Type a short description. Choose <i>Add Feed</i> in the Outliner menu, and paste the URL of an RSS or Atom feed. Repeat this for all the feeds you want to add to your list.</p></li> <li><p>If you want to include another OPML list in this list, press Return to add a new headline. Type a short description, then choose <i>Add Include</i> in the Outliner menu, and enter the URL an OPML file. When River4 processes the list, it will be as if all the feeds in that list were in this list. Here's a Fargo docs <a href="http://fargo.io/docs/fargo/includes.html">page</a> on includes. </p></li> <li><p>You can use the include feature in the previous step to create a placeholder list in your lists folder. Just have it include the list you're editing in Fargo. You can get the URL of the outline by choosing Get Public Link in Fargo's File menu. This is a bit complicated, but worth studying and understanding because it makes Fargo a simple way to tell River4 what feeds you want it to read. </p></li> </ol> <h4>Managing multiple rivers</h4> <p>It's easy to have a second or third river, just create another OPML file in <i>lists</i> folder. It's probably good to get started with a single list, and get a feel for how busy the feeds are, how many new items you get every time you look at the river.</p> Tue, 04 Aug 2015 16:14:38 GMT http://river4.smallpict.com/2015/08/04/editingSubscriptionLists.html Installing River4 on Ubuntu http://river4.smallpict.com/2015/08/04/installingRiver4OnUbuntu.html <p>This is a checklist I've used for installing "River4" on a fresh Ubuntu v14.04 server. </p> <h4>Install node.js</h4> <pre>sudo apt-get update sudo apt-get install nodejs sudo apt-get install npm sudo apt-get install nodejs-legacy</pre> <p>We also install <a href="https://www.npmjs.com/">npm</a>, a requirement to run Node apps. </p> <p><a href="https://packages.debian.org/sid/nodejs-legacy">nodejs-legacy</a> makes it possible to run apps by saying <i>node app.js</i> instead of having to use nodejs, an oddity of Ubuntu.</p> <h4>Install forever</h4> <pre>sudo npm install forever -g</pre> <p>There are lots of ways to get apps to launch in the background, I like <a href="https://github.com/foreverjs/forever">forever</a> because it keeps the app running even if it crashes. River4, of course, never crashes (heh) but you never really know. <img title=':wink:' alt=':wink:' class='emoji' src='http://fargo.io/code/emojify/images/emoji/wink.png' align='absmiddle' /></p> <h4>Install git</h4> <pre>sudo apt-get install git</pre> <p>I like to install git, because it makes it easy to install River4 from GitHub.</p> <h4>Install River4</h4> <pre>git clone https://github.com/scripting/river4.git</pre> <p>That should create a directory containing River4 at /home/ubuntu/river4</p> <p>To run it, follow the instructions on the <a href="http://river4.smallpict.com/2015/07/20/usingRiver4WithFilesystemStorage.html">River4 howto</a>. </p> Tue, 04 Aug 2015 14:10:16 GMT http://river4.smallpict.com/2015/08/04/installingRiver4OnUbuntu.html Video: Installing River4 on a Mac http://river4.smallpict.com/2015/07/20/videoInstallingRiver4OnAMac.html <p>A 15-minute video showing <a href="http://river4.smallpict.com/2015/07/20/usingRiver4WithFilesystemStorage.html">how to install</a> River4 on a Mac system. </p> <iframe width="640" height="480" src="https://www.youtube.com/embed/2wIs2gkFmWk" frameborder="0" allowfullscreen></iframe> <h4>Screen shots</h4> <p>I promised at the end of the video to upload a few screen shots of what the river looks like after it's been running a while.</p> <ol> <li><p>The <a href="http://scripting.com/2015/07/20/dashboard.png">dashboard</a> after running for 38 minutes.</p></li> <li><p>The <a href="http://scripting.com/2015/07/20/nbaNews.png">NBA news</a> tab at that time. </p></li> <li><p>The <a href="http://scripting.com/2015/07/20/nytriver.png">NYT news</a> tab. </p></li> <li><p>A <a href="https://dl.dropboxusercontent.com/u/36518280/misc/river4data2.zip">snapshot</a> of my river4data folder after running for 90 minutes.</p></li> </ol> <h4>Facebook version of video</h4> <p>I also uploaded it <a href="https://www.facebook.com/video.php?v=367101606830584&amp;set=vb.100005922431565&amp;type=2&amp;theater&amp;notif_t=video_processed">to Facebook</a> and think it's a bit higher quality. </p> Mon, 20 Jul 2015 14:11:05 GMT http://river4.smallpict.com/2015/07/20/videoInstallingRiver4OnAMac.html Using River4 with filesystem storage http://river4.smallpict.com/2015/07/20/usingRiver4WithFilesystemStorage.html <p>These instructions show you how to install "River4" on a machine that's never had Node running on it, using the file system for storage. You can also use S3 for storage, and there's a separate <a href="http://river4.smallpict.com/2014/10/17/usingRiver4WithS3Storage.html">howto</a> for that kind of installation. </p> <p>These instructions assume you're installing on a Macintosh, but they're very close to installing on a Unix or Windows machine. </p> <h4>How to</h4> <ol> <li><p>Download Node.js from <a href="http://nodejs.org/">this page</a>. Get the Macintosh installer. You don't need the source code.</p></li> <li><p>Run the pkg file you downloaded. Accept all defaults, install for all users. At the end it tells you where everything was installed. I didn't need this information. It also says to make sure /usr/local/bin is in your $PATH. I didn't need to do anything, apparently it was setup correctly by default. </p></li> <li><p><a href="https://github.com/scripting/river4/archive/master.zip">Download</a> River4 from the <a href="https://github.com/scripting/river4">GitHub repository</a>, and create a folder (it can be anywhere). Copy all the files into that folder. Using the shell <i>cd</i> command, make that folder the current directory. </p></li> <li><p>Install the packages River4 needs.<div class="divUnixLine">npm install</div></p></li> <li><p>Launch River4:<div class="divUnixLine">node river4.js</div></p></li> <li><p>Shortly after it launches, River4 automatically creates a <i>river4data</i> folder in the same folder as the River4 app. It contains sub-folders: data, lists and rivers. </p></li> <li><p>For now, <i>lists</i> is the important folder. Copy a few OPML subscription lists into the <i>lists</i> folder. If you need some to help test your setup, you can download some examples <a href="http://river4.smallpict.com/2014/09/20/exampleSubscriptionLists.html">here</a>. Ultimately you'll want to create and maintain your own. "Fargo" is very good for that, its native file format is OPML. Use the Add Feed command in the Outliner menu. </p></li> <li><p>Let River4 run for a while. As soon as there are new items in the feeds in your lists, river files will show up in the rivers2 folder, one corresponding to each of your lists. These are used by river browser software to display the new items for readers. See the next section.</p></li> </ol> <h4>Viewing your rivers</h4> <p>River4 is also a web server, running on port 1337. </p> <p>If you go to the home page of the server, you'll see the contents of your rivers, and commands that take you to the dashboard, repository, mail list, this blog.</p> <p><a href="http://localhost:1337/"><code>http://localhost:1337/</code></a></p> <h4>Watch a video</h4> <p>Here's a <a href="http://river4.smallpict.com/2015/07/20/videoInstallingRiver4OnAMac.html">15-minute video</a> where I install a River4, famous-chef style. <img title=':blush:' alt=':blush:' class='emoji' src='http://fargo.io/code/emojify/images/emoji/blush.png' align='absmiddle' /></p> <p>It includes screen shots of the River4 dashboard and home page after 38 minutes, and a copy of a <i>river4data</i> folder after 90 minutes. </p> <h4>Join the mail list</h4> <p>If you're going to run River4, I highly recommend joining the <a href="https://groups.google.com/forum/?fromgroups#!forum/river4">River4 mail list</a>. </p> Mon, 20 Jul 2015 12:11:23 GMT http://river4.smallpict.com/2015/07/20/usingRiver4WithFilesystemStorage.html JSON encoding issue fixed http://river4.smallpict.com/2015/06/16/jsonEncodingIssueSolved.html <p>Andrew Shell <a href="https://groups.google.com/forum/?fromgroups#!topic/river4/BfjMIoMyyiA">reported</a> an issue with breakage in JSON in a relatively recent version of JavaScript. Kind of disturbing that this kind of breakage can actually happen, but we're happy to have a fix.</p> <p>In "River4" v0.114, I did more than Andrew recommends, and replaced all the calls to JSON.stringify in River4 with calls to the utility routine jsonStringify, and then made the fix inside that routine. </p> <p>I've tested this with my three main rivers (two River4 installations) and it seems to work. </p> Tue, 16 Jun 2015 15:58:40 GMT http://river4.smallpict.com/2015/06/16/jsonEncodingIssueSolved.html New way to configure River4 http://river4.smallpict.com/2015/05/10/newWayToConfigureRiver4.html <p>New in "River4" v0.110.</p> <p>I had to switch from <a href="http://liveblog.co/users/davewiner/2015/05/09/explainingWhatHappenedWithHeroku.html">Heroku</a> to an AWS-hosted Ubuntu server that's hosting a bunch of other apps. Without the isolation that Heroku provides, the <a href="http://liveblog.co/users/davewiner/2015/05/08/deathToEnvironmentParameters.html">environment variables</a> of all these apps were getting in each others' way. Rather than hack my way through all the arcane rules of environment variables, I decided to create a simpler more reliable (imho) way to configure "River4" and "nodeStorage", using a config.json file in the same directory as river4.js.</p> <h4>What's in config.json</h4> <p>Here's a list of values you can include in this file:</p> <ol> <li><p>fspath</p></li> <li><p>password</p></li> <li><p>s3path</p></li> <li><p>PORT</p></li> <li><p>s3defaultAcl</p></li> </ol> <p>They work exactly the same way as the environment variables with the same names.</p> <p>If you specify an environment variable and an element in config.json, the one in config.json takes precedence. </p> <h4>Three exceptions</h4> <p>If you're using S3 storage, you need to provide the three values for your AWS account. Since the Amazon library is looking for these in environment variables, you must provide them that way. </p> <ol> <li><p>AWS_ACCESS_KEY_ID</p></li> <li><p>AWS_SECRET_ACCESS_KEY</p></li> <li><p>AWS_REGION</p></li> </ol> <h4>Example</h4> <p>Here's an example, the contents of config.json on my server (with values changed).</p> <p><img src="http://scripting.com/2015/05/10/config.png" width="300" height="79" border="1" alt="A picture named config.png"></p> Sun, 10 May 2015 12:41:40 GMT http://river4.smallpict.com/2015/05/10/newWayToConfigureRiver4.html What's in the River4 folders? http://river4.smallpict.com/2015/05/05/whatsInTheRiver4Folders.html <p>There are three top-level folders: data, lists and rivers. </p> <p>The only one folder that contains input is the lists folder. Everything else is created and maintained by the River4 software. </p> <p>The data folder has several sub-folders.</p> <ol> <li><p>calendar is a calendar-structured folder broken down by year and month. Each file is a day's worth of news gathered from the feeds all your lists subscribe to. </p></li> <li><p>feeds has a sub-folder for every feed your river is following. There's only one file in each sub-folder, but there's room for growth. </p></li> <li><p>feedsInLists.json is a list of feeds, and a reference count for each feed. If the count goes to zero, we don't have to read the feed, because there aren't any lists subscribed to it. </p></li> <li><p>feedsStats.json is a JSON array containing information about each of the feeds. </p></li> <li><p>lists has a sub-folder for each of the lists. There's one file in each folder with info about the list. </p></li> <li><p>prefsAndStats.json stores preference settings and overall stats for your River4 installation. </p></li> <li><p>riversArray is an array of information about your rivers. You can edit the title and description for each river to control the display of the home page. </p></li> </ol> <h4>The meaning of "enabled"</h4> <p>When you see a JSON file that has an element called enabled, if you set it false, the software will stop processing that object. So if you set the enabled element of a listInfo.json file to false, we will stop reading the list. It's useful if you want to keep everything in place, but turn off parts of the server.</p> <p>You can turn the whole River4 aggregator and server off by setting enabled false in prefsAndStats.json.</p> Tue, 05 May 2015 14:13:11 GMT http://river4.smallpict.com/2015/05/05/whatsInTheRiver4Folders.html Found a serious River4 problem http://river4.smallpict.com/2015/04/21/foundASeriousRiver4Problem.html <p>I've seen this behavior before, at midnight, I end up with an empty river, but after the next read the river is back to normal. I now know why this happens, because it showed up in podcatch.com, and there the problem takes longer to clear, because it takes longer for a new item to show up in the river.</p> <p>The problem is in <a href="https://github.com/scripting/river4/blob/master/river4.js#L1063">buildOneRiver</a>. It starts by calling <a href="https://github.com/scripting/river4/blob/master/river4.js#L1239">doOneDay (starttime)</a>, where starttime is the current time. There is no file in the /data/calendar/ folder for today. So the first read fails. When a read fails, it calls finishBuild (). Since we haven't seen any data yet, the river is empty. So an empty river is written and buildOneRiver returns.</p> <p>It could be total disaster if there ever was a day where nothing showed up. Not only would the river stay empty all day, but it would alway stop when it hit the empty day, making all data before that date basically unreachable.</p> <p>An easy workaround for now was to create an empty array for today. When I did that and rebuilt the river, all the data showed up again on "podcatch.com". <i>Whew.</i></p> <p>This error probably showed up every night and wasn't cleared until the first podcast of the morning showed up. I never saw it because I'm generally not looking at podcatch.com after midnight. But other people were certainly seeing it.</p> <p>We're going to need a better fix for this, but first I wanted to be sure to document the problem. Having done so I can now go to sleep.</p> <p>Good night. <img title=':wink:' alt=':wink:' class='emoji' src='http://fargo.io/code/emojify/images/emoji/wink.png' align='absmiddle' /></p> <p>Dave</p> <p>PS: Thanks to the Pelicans and the Warriors. They were playing such an excellent <a href="http://www.nytimes.com/aponline/2015/04/21/sports/basketball/ap-bkn-pelicans-warriors.html?partner=rss&amp;emc=rss">game</a>, that's why I was farting around with the computers after midnight, during a commercial break during the game. </p> Tue, 21 Apr 2015 05:22:16 GMT http://river4.smallpict.com/2015/04/21/foundASeriousRiver4Problem.html River4 needs debugging http://river4.smallpict.com/2015/02/22/river4NeedsSomeDebugging.html <p>"River4" is pretty solid software, but it needs more attention to get it all the way. </p> <h4>What's going wrong</h4> <p>I've been seeing some new, not good, behavior on my "River4" installation in the last month or so. First it started going deaf, not responding to HTTP requests, causing my serverMonitor app to send emails saying the server was down. I'd get 20 or 30 of these a day. And the new items would take longer to show up after this started happening. It seemed as if the server was thrashing. Some operation that used to be quick was now taking a long time.</p> <h4>A new instance</h4> <p>So I decided to create a new instance on Heroku that read the same lists, and let it run for a while (a week or so). It was running smoothly so I just switched domains, and shut the old one down. It was gratifying how easy this was to do. That was a couple of weeks ago.</p> <p>Now the new instance is starting to misbehave.</p> <p>Sometimes it renders one of the river files with just items from one feed. It corrects itself on the next run. It's annoying, it should be found and fixed, whatever is causing this, but I can live with it.</p> <p>Then a couple of days ago, it stopped finding new items in feeds, for five hours. I rebooted the server, and that fixed it. Until it happened again this morning. Last new item in all my rivers was at approx 3AM. A reboot of the server seems to have cured it.</p> <h4>Help trap the problems?</h4> <p>I'm very focused right now on shipping a new end-user product, so I can't detour to figure out what's going on here.</p> <p>If anyone has the time to trap these problems, I'd be happy to try to fix them, once I can swing back around to River4. Probably within a few weeks. </p> <p>I am sharing my subscription lists so other people can run a test setup with the same feeds as mine. It's possible that I'm pushing River4 harder than anyone else. So maybe I'm seeing problems you all will be seeing soon. Or maybe you are already seeing them? </p> <h4>Links</h4> <p>Here's my <a href="http://radio3.io/rivers/">rivers page</a>, so you can see what I'm reading. </p> <p>I've zipped up my <a href="https://dl.dropboxusercontent.com/u/36518280/misc/myRiver4Lists.zip">current set</a> of subscription lists.</p> <p>Discuss on the <a href="https://groups.google.com/forum/#!topic/river4/3tG3S9li3yU">River4 list</a>.</p> Sun, 22 Feb 2015 16:50:26 GMT http://river4.smallpict.com/2015/02/22/river4NeedsSomeDebugging.html River4 console v0.41 http://river4.smallpict.com/2014/12/14/river4ConsoleV041.html <ol> <li><p>If you added a feed that did not have a description, saving would fail. </p></li> <li><p>I was having a hell of a time getting the left-margin icons to respond to clicks. There's some weird interaction going on. If anyone has a clue, please post a comment. In the meantime, as a workaround, I added a new <i>Feed</i> menu to the menu bar with three commands that do what the icons do. </p></li> </ol> Sun, 14 Dec 2014 16:45:47 GMT http://river4.smallpict.com/2014/12/14/river4ConsoleV041.html New River4, new console app http://river4.smallpict.com/2014/12/05/newRiver4NewDashboardApp.html <p>The big new <a href="http://scripting.com/2014/12/05/consoleScreen.png">addition</a> is the <a href="http://river4.io/">River 4 Console</a>. </p> <p>You can edit subscription lists for your River4 installation in a browser, using the Concord outliner, the same one that's at the core of Fargo. </p> <p>You can edit them either on the server machine, or remotely. </p> <h4>How to</h4> <ol> <li><p>Install the latest version of "River4", v0.108, on your server. You can <a href="https://github.com/scripting/river4/archive/master.zip">download</a> it from the GitHub repository.</p></li> <li><p>If you want to access the server remotely using the new dashboard, set an environment variable, <i>password.</i> If you don't set the password, you will only be able to access the server through localhost.</p></li> <li><p>Go to <a href="http:/river4.io/">river4.io</a>. In the <i>Server</i> menu, choose <i>Settings. </i> </p></li> <li><p>Enter the address of your server, and the password you set in step 2. <a href="http://scripting.com/2014/12/05/serverdsettings.png">Screen shot</a>.</p></li> <li><p>Reload river4.io to get the data from your server. </p></li> </ol> <h4>How edit a list</h4> <p>You can add a new item to the list by clicking the + icon in the left margin. Enter the URL of the feed. If it could be read, the feed will appear in the list. </p> <p>Click the Save button to send the changed list back to the server.</p> <p>Choose any of the items in the Lists menu to load your list in the outliner box.</p> <h4>Other features</h4> <p>In the Server menu, you can also disable the server, and re-enable it. You can open the dashboard, to watch the server as its running (replacing the original dashboard, which still works of course). And you can quickly view the river that is derived from the list you're editing.</p> <p>In the left margin, you can edit the attributes of any headline in a subscription list, or view the feed in a special XML browsing window (avoiding the crappy way most browsers deal with RSS).</p> <h4>What's new in River4 0.108</h4> <p>Basically, a bunch of new server endpoints to support editing of lists and prefs.</p> <h4>This is a pausing point</h4> <p>Originally, I planned to package this as a node-webkit app before releasing it. A couple of days ago I realized this was a bigger undertaking than I had imagined, and wanted to pause for a while here, to shake the bugs out, and learn how this works, before locking it down as a standalone app. </p> <p>The goal remains, to make River4 more of an end-user experience. This release makes it easy to edit subscription lists, monitor the server, and turn off the server and turn it back on. A lot of the work toward reaching the goal is in this release. I plan to use it myself to manage my own River4 server, and encourage others to do the same.</p> Fri, 05 Dec 2014 05:06:09 GMT http://river4.smallpict.com/2014/12/05/newRiver4NewDashboardApp.html Using Dropbox and River4 http://river4.smallpict.com/2014/11/10/usingDropboxAndRiver4.html <p>A few notes I <a href="https://groups.google.com/forum/?fromgroups=#!topic/river4/efbUzFItmHg">posted</a> to the River4 mail list.</p> <ol> <li><p>Jake Savin worked as a developer at UserLand, and I'm really glad he's using River4 now. We know each other pretty well, having worked together on Manila and Radio UserLand, which included the first in the series of Rivers culminating in River4.</p></li> <li><p>This is a different way of doing it from any of the previous products, but then I realized -- it's not. You can use River4 in exactly the same way we used <a href="http://river3.opml.org/">River3</a> with Dropbox, and it would work really well in this mode. </p></li> <li><p>To do so, install River4 on a local machine, and <a href="http://river4.smallpict.com/2014/09/25/bareBonesRiver4Howto.html">configure</a> it to read and write from a Dropbox-shared folder. </p></li> <li><p>To edit one of your subscription lists, you can use the OPML Editor, because it's designed to read and write files on the local OS. </p></li> <li><p>And you can include the "River.js" files in your pages, by getting a public link to the files. No need to operate Apache, or any public-facing server at all.</p></li> </ol> <p>I just wanted to get this out there, maybe it'll make someone's life easier. <img title=':wink:' alt=':wink:' class='emoji' src='http://fargo.io/code/emojify/images/emoji/wink.png' align='absmiddle' /></p> Mon, 10 Nov 2014 18:11:29 GMT http://river4.smallpict.com/2014/11/10/usingDropboxAndRiver4.html Update to the river renderer http://river4.smallpict.com/2014/11/03/updateToTheRiverRenderer.html <p>Now if an &lt;source:outline> has a flMarkdown attribute with the value false, we'll render it as a structured outline. </p> <p>This is the same <a href="http://fargo.io/blog/2014/07/12/fargo164.html">convention</a> that Fargo uses, so if you're using Fargo, your web page will look great, and so will the outline in the river.</p> <p>Another loose-end tied off. <img title=':wink:' alt=':wink:' class='emoji' src='http://fargo.io/code/emojify/images/emoji/wink.png' align='absmiddle' /></p> Tue, 04 Nov 2014 00:12:08 GMT http://river4.smallpict.com/2014/11/03/updateToTheRiverRenderer.html River4 v0.99 http://river4.smallpict.com/2014/11/01/river4V099.html <p>Fixing problems with running <a href="https://github.com/scripting/river4#v099----11114-by-dw">River4</a> with file system storage.</p> <ol> <li><p>Only read lists whose names end with .opml. We were trying to read an invisible file on the Mac, .DS_Store. </p></li> <li><p>When running in local file system mode, on Windows, there are more illegal characters than were previously tested for. Now we test for them. See this <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx">page</a> on the Windows developer site for a list. </p></li> </ol> Sat, 01 Nov 2014 20:14:56 GMT http://river4.smallpict.com/2014/11/01/river4V099.html Using River4 with S3 storage http://river4.smallpict.com/2014/10/17/usingRiver4WithS3Storage.html <p>These instructions show you how to set up <a href="https://github.com/scripting/river4">River4</a> to work with Amazon S3 storage.</p> <h4>What you'll need</h4> <ol> <li><p>A node.js installation.</p></li> <li><p>An Amazon account, and an S3 bucket to store the JSON files, and a small HTML file.</p></li> <li><p>One or more OPML subscription list files.</p></li> </ol> <h4>How to install</h4> <ol> <li><p>Create an S3 bucket to hold all your subscription lists, rivers, and data for the aggregator. </p></li> <li><p>On the node.js system, set an environment variable, s3path, to contain the path to the bucket created in step 1.<div class="divUnixLine">export s3path=/river.mydomain.com/</div></p></li> <li><p>Again, on the node.js system, set the two AWS environment variables. This allows the River4 app to write to your bucket.<div class="divUnixLine">export AWS_ACCESS_KEY_ID=12345<br>export AWS_SECRET_ACCESS_KEY=TUVWXYZ</div></p></li> <li><p>Launch river4.js on a node.js system. Suppose that server is aggregator.mydomain.com.</p></li> <li><p>Look in the bucket. You should see a data folder, with a single file in it containing the default value of prefs and stats for the app. There's also an index.html file, which will display your rivers in a simple way, providing code you can crib to create your own way of browsing (room for improvement here, for sure).</p></li> <li><p>Create a folder at the top level of the bucket called "lists". Save one or more OPML subscription lists into that folder.</p></li> <li><p>After a while you should see a new folder called "rivers" created automatically by the software. In that folder you should see one JSON file for each list. It contains the news from those feeds, discovered by River4. This format is designed to plug into the beautfiul" river displayer. </p></li> <li><p>If you want to watch the progress of the aggregator, you can view this page. <div class="divUnixLine">http://aggregator.mydomain.com/serverdata</div></p></li> </ol> <h4>Notes</h4> <ol> <li>When you set up your S3 bucket, make sure that web hosting is enabled and index.html is the name of your index file. Here's a <a href="http://static.scripting.com/larryKing/images/2014/06/01/bucketSetup.gif">screen shot</a> that shows how to set it up. </li> </ol> Fri, 17 Oct 2014 14:26:19 GMT http://river4.smallpict.com/2014/10/17/usingRiver4WithS3Storage.html The Hello World of rivers http://river4.smallpict.com/2014/10/05/theHelloWorldOfRivers.html <p>Suppose you have "River4" running, generating a few "River.js" files, and you want to include one in your news site or blog. That's why I put this simple example page together, to provide source code you can crib to do the job.</p> <h4>The source</h4> <p>Here's a <a href="https://gist.github.com/scripting/87903653a0f5f6df13b4">gist</a> that contains the source of an <a href="http://fargo.io/code/helloriver/">HTML page</a> that you can crib from. </p> <p>The code on the Hello River page is yours to do with as you please. But the files it includes are copyrighted and not at this time available under an open source license. </p> <h4>How to</h4> <p>Include river.js and river.css in your page. It automatically includes the files it needs.</p> <p>The other files we include, the Lora font, menus.css, are just to make the sample app look good. They aren't required for your pages.</p> <p>To load a river, call httpGetRiver with the url of the <a href="http://riverjs.org/">River.js</a> file as its only parameter.</p> <p>The commands in the Rivers menu loads a few of the rivers that my server maintains.</p> <h4>Update #1 -- Oct 12, 2014</h4> <p>I needed to remove the inclusion of jQuery and Bootstrap from river.js, so the river code could be included in Fargo's rendered pages. This means the Hello World app changed, and your code will have to change as well. </p> <p>Just add the three lines in the &lt;head> section of your page as the <a href="https://gist.github.com/scripting/87903653a0f5f6df13b4">example</a> does.</p> <p>&lt;script src="http://fargo.io/code/jquery-1.9.1.min.js"></script></p> <p>&lt;link href="http://fargo.io/code/bootstrap.css" rel="stylesheet"></p> <p>&lt;script src="http://fargo.io/code/bootstrap.min.js"></script></p> <h4>Update #2 -- Oct 12, 2014</h4> <p>I removed the top and bottom margin from .divRiverContainer. It really should be up to the application that includes the river to determine what space is left above and below the river.</p> Sun, 05 Oct 2014 16:39:20 GMT http://river4.smallpict.com/2014/10/05/theHelloWorldOfRivers.html Using River4 with file system storage http://river4.smallpict.com/2014/09/25/bareBonesRiver4Howto.html <p><i><b>Important: There's a <a href="http://river4.smallpict.com/2015/07/20/usingRiver4WithFilesystemStorage.html">new version</a> of these instructions that are easier, because the product has improved. These instructions are being maintained as an archive. </b></i></p> <p>These instructions show you how to install "River4" on a machine that's never had Node running on it, using the file system for storage. You can also use S3 for storage, and there's a separate <a href="http://river4.smallpict.com/2014/10/17/usingRiver4WithS3Storage.html">howto</a> for that kind of installation. </p> <p>These instructions assume you're installing on a Macintosh, but they're very close to installing on a Unix or Windows machine. </p> <h4>How to</h4> <ol> <li><p>Download Node.js from <a href="http://nodejs.org/download/">this page</a>. Get the Macintosh installer. You don't need the source code.</p></li> <li><p>Run the pkg file you downloaded. Accept all defaults, install for all users. At the end it tells you where everything was installed. I didn't need this information. It also says to make sure /usr/local/bin is in your $PATH. I didn't need to do anything, apparently it was setup correctly by default. </p></li> <li><p><a href="https://github.com/scripting/river4/archive/master.zip">Download</a> River4 from the <a href="https://github.com/scripting/river4">GitHub repository</a>, and create a folder (it can be anywhere). Copy all the files into that folder. The only two that are actually needed are river4.js and package.json, so if you want to save some space, you can delete the others. Using the shell <i>cd</i> command, make that folder the current directory. </p></li> <li><p>Install the packages River4 needs.<div class="divUnixLine">npm install</div></p></li> <li><p>I took a deep breath, reviewed everything to be sure I set it up correctly, and launched River4:<div class="divUnixLine">node river4.js</div></p></li> </ol> <h4>Accessing the dashboard</h4> <p>You can access the dashboard on the local computer with this URL, entered in a browser:</p> <p><a href="http://localhost:1337/dashboard">http://localhost:1337/dashboard</a></p> <h4>Join the mail list</h4> <p>If you're going to run River4, I highly recommend joining the <a href="https://groups.google.com/forum/?fromgroups#!forum/river4">River4 mail list</a>. </p> Thu, 25 Sep 2014 15:03:01 GMT http://river4.smallpict.com/2014/09/25/bareBonesRiver4Howto.html River4 and the local filesystem http://river4.smallpict.com/2014/09/24/river4WorksWithLocalFilesystem.html <p>In previous versions, "River4" stored all its data in an S3 bucket. </p> <p>Starting in v0.96, it can be configured to store its data in the local filesystem. </p> <p>The new version is on <a href="https://github.com/scripting/river4">GitHub</a>.</p> <h4>How it works</h4> <p>There's a new environment variable called <i>fspath.</i></p> <p>If it's not defined, everything is exactly as before.</p> <p>If it is defined, we use a local directory for storage. It's specified by fspath.</p> <h4>Example</h4> <p>I created a folder on my desktop computer, and set the environment variable with this command in the Unix shell:</p> <pre>export fspath=/Users/davewiner/river4data/</pre> <p>I created a <i>lists</i> sub-folder, and copied a few OPML <a href="http://river4.smallpict.com/2014/09/20/exampleSubscriptionLists.html">subscription lists</a> in the folder. </p> <p>I copied river4.js into another folder and used the <i>cd</i> command to change to that directory, and entered:</p> <pre>node river4.js</pre> <p>The server booted up and started reading feeds. </p> <p>PS: Of course I already had node.js installed, and had installed feedparser and opmlparser.</p> <h4>Accessing the dashboard</h4> <p>You can access the dashboard on the local computer with this URL, entered in a browser:</p> <p><a href="http://localhost:1337/dashboard">http://localhost:1337/dashboard</a></p> <h4>How it's implemented</h4> <p>I wrote a new set of "fs" routines that are parallel to the s3 routines, and then added a layer of abstraction. Each routine calls the s3 routine if it's set up that way and calls the fs routine if it's set up the other way. </p> <p>Fairly straightforward, and it worked the first time I ran it. </p> <p>I have it running on two systems, one on a Mac desktop machine, storing data in the file system, and my original system in Heroku using S3 storage. Same software, configured differently.</p> <h4>Pointers</h4> <p><a href="http://scripting.com/2014/09/25/#a1411650541">A blog post</a>, providing perspective.</p> Thu, 25 Sep 2014 02:30:57 GMT http://river4.smallpict.com/2014/09/24/river4WorksWithLocalFilesystem.html Example subscription lists http://river4.smallpict.com/2014/09/20/exampleSubscriptionLists.html <p>Getting started with a new River4 installation can be slow going if you don't have a good set of subscription lists to start with. </p> <p>To help get the process going for new users, here are a few of the lists I use, in a <a href="https://dl.dropboxusercontent.com/u/36518280/outlines/subscriptionLists.zip">zip archive</a>. </p> Sun, 21 Sep 2014 01:25:06 GMT http://river4.smallpict.com/2014/09/20/exampleSubscriptionLists.html Using RiverBrowser with Radio3 http://river4.smallpict.com/2014/09/20/usingRiverbrowserWithRadio3.html <p>Each item in "RiverBrowser" has a tiny gray <i>share</i> icon in the lower left corner of the item display. This icon acts just like the Radio3 bookmarklet. Click it, and Radio3 will come to the front, with the contents of this post populating the dialog. </p> <p><img src="http://static.scripting.com/larryKing/images/2014/09/20/shareIcon.gif" width="662" height="238" border="0" style="padding-left: 25px; padding-bottom: 10px; padding-top: 10px; padding-right: 15px;" alt="A picture named shareIcon.gif"></p> Sat, 20 Sep 2014 14:51:44 GMT http://river4.smallpict.com/2014/09/20/usingRiverbrowserWithRadio3.html Using RiverBrowser with River4 http://river4.smallpict.com/2014/09/20/usingRiverbrowserWithRiver4.html <p>If you're running the open source "River4" aggregator, you can configure "RiverBrowser" to access your <a href="http://riverjs.org/">river</a> files. </p> <ol> <li><p>You must enable CORS in the S3 bucket that River4 runs from. Here's a copy of the <a href="https://gist.github.com/scripting/afc95f1b951978c58ad7">configuration</a> I used for my bucket. </p></li> <li><p>You must be signed on to Twitter. We only use Twitter for the ID system, RiverBrowser won't post anything to Twitter, or access any of your messages or private data. The command to sign on is in the system menu, at the right end of the menu bar.</p></li> <li><p>Choose <i>My rivers</i> in the Rivers menu. An <a href="http://static.scripting.com/larryKing/images/2014/09/20/myRivers.gif">attribute editor</a> pops up, with all the rivers that are currently being displayed by RiverBrowser. You can add or delete items from the list, or change the names of any of the items. This is where you can add pointers to your own rivers. </p></li> </ol> Sat, 20 Sep 2014 14:45:10 GMT http://river4.smallpict.com/2014/09/20/usingRiverbrowserWithRiver4.html What is RiverBrowser? http://river4.smallpict.com/2014/09/20/whatIsRiverbrowser.html <p>"RiverBrowser" is the reader-interface for rivers produced by the "River4" aggregator. River4 has new features that support output that's produced by both "Fargo" and "Radio3". </p> <p>All these products are designed to work with each other, and support innovation across all the products. The connections are all open, and all the parts are replaceable. You can generate compatible feeds with any blogging environment or content management system. The extensions to RSS we're using are all documented in the new source namespace. Any aggregator can produce the JSON files that River4 generates, that are designed to plug into RiverBrowser. </p> <p>If you've been following my development process, this is the cap, where it all comes together in a simple neat package anyone can use. You don't have to understand all of the tools behind it, you don't have to understand <i>any</i> of them, actually. It's just useful and usable as-is</p> Sat, 20 Sep 2014 14:44:47 GMT http://river4.smallpict.com/2014/09/20/whatIsRiverbrowser.html River4 v0.94 http://river4.smallpict.com/2014/08/06/river4V094.html <p>Andrew Shell dug in and found the source of a problem in building rivers.</p> <p>If River4 started to re-load the OPML reading lists while a build was going on, it could cause the rivers to skip items that shouldn't be skipped. The problem would correct itself quickly on my server (made it hard to track down the problem), not so quickly on Andrew's. </p> <p>The problem was pretty easy to fix, after reading <a href="https://groups.google.com/forum/#!msg/river4/6BOz2raCcRs/296_QEXzFzQJ">Andrew's thorough description</a>.</p> <p>I had to arrange things so that the list-reading happens after the rivers are built. I put a one minute cushion in there. Have a look at everyFiveMinutes () and everyMinute () to see how it was done.</p> <p>The new version is on the <a href="https://github.com/scripting/river4">River4 repository</a>.</p> Wed, 06 Aug 2014 14:19:30 GMT http://river4.smallpict.com/2014/08/06/river4V094.html River4 v0.88 -- includes in lists http://river4.smallpict.com/2014/06/17/river4V088.html <p>A "River4" subscription list can now contain OPML <a href="http://fargo.io/docs/fargo/includes.html">include</a> nodes, enabling lists of lists, a crucial feature for a modern RSS reader app.</p> <p>This is nice when you want to let a group of people collaborate on the curation of a river.</p> <p>Also makes it possible to use "Fargo" to edit subscription lists. Just edit the list in the S3 bucket to contain a node that includes the Fargo list. The editors don't need to have access to the bucket.</p> <h4>Use-case</h4> <p>I needed this feature for a project I'm doing where the curator is comfortable editing OPML lists, but isn't ready to make the investment in learning how to operate a river. </p> <p>It also makes it possible to switch the river to another server without changing any update procedures. </p> <h4>Limit</h4> <p>We only go two levels deep. That is you can have an include in the top-level subscription list, but any includes in included lists are ignored.</p> <p>The only reason for having this limit is that it eliminates the need to have a stack to make sure there's no circular recursion. </p> Tue, 17 Jun 2014 21:20:59 GMT http://river4.smallpict.com/2014/06/17/river4V088.html